2024-04-25T00:12:41.655 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:12:41.655 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254 --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/orig.config.yaml 2024-04-25T00:12:41.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:12:41.789 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672254' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDxZRLolfEPe1NQho8iO0ZiTpwP2TOQflTynmPJxZDdWG22YaIqvUVpRh4nugrJb8LFPm9JX0ZRKOK7GGwQwD04= smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYCAqdyd/My8yRDVW7jMMwZFZE+iYb27zKMyjPF8kB15/hD4+MWZXBFmasCymgJmFKe/BETiivh0u3McDw3lmI= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHsU6KTZPWccAp5ZpL/eEc0D+Ij6ue9nRjFTr7NoaB69+/liUP8QdYAamhajNRtbZAbVOi3lUUvYNWYEpcLc+RI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:12:41.790 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:12:41.790 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:12:41.790 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:12:41.792 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:12:41.813 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:12:41.814 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:12:41.814 INFO:teuthology.packaging:ref: None 2024-04-25T00:12:41.814 INFO:teuthology.packaging:tag: None 2024-04-25T00:12:41.814 INFO:teuthology.packaging:branch: squid 2024-04-25T00:12:41.814 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:12:41.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T00:12:42.011 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T00:12:42.012 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:12:42.059 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:12:42.059 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:12:42.066 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:12:42.084 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:12:42.088 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:12:42.112 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:07:41.200321', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDxZRLolfEPe1NQho8iO0ZiTpwP2TOQflTynmPJxZDdWG22YaIqvUVpRh4nugrJb8LFPm9JX0ZRKOK7GGwQwD04='} 2024-04-25T00:12:42.135 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:07:41.201872', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYCAqdyd/My8yRDVW7jMMwZFZE+iYb27zKMyjPF8kB15/hD4+MWZXBFmasCymgJmFKe/BETiivh0u3McDw3lmI='} 2024-04-25T00:12:42.152 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:07:41.203071', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHsU6KTZPWccAp5ZpL/eEc0D+Ij6ue9nRjFTr7NoaB69+/liUP8QdYAamhajNRtbZAbVOi3lUUvYNWYEpcLc+RI='} 2024-04-25T00:12:42.152 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:12:42.157 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T00:12:42.157 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T00:12:42.157 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T00:12:42.157 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:12:42.246 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f99decab160>, signals=[15]) 2024-04-25T00:12:42.246 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:12:42.254 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:12:42.254 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:12:42.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:12:42.333 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T00:12:42.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:12:42.406 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:12:42.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:12:42.481 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:12:42.488 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-04-25T00:12:42.506 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-04-25T00:12:42.507 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:12:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:12:42.566 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:12:42.566 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:12:42.566 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:12:42.566 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:12:42.566 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-25T00:12:42.585 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-25T00:12:42.604 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-25T00:12:42.604 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:12:42.662 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:12:42.663 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:12:42.663 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:12:42.663 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:12:42.663 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:12:42.663 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-25T00:12:42.685 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-25T00:12:42.703 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-25T00:12:42.703 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:12:42.761 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:12:42.761 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-25T00:12:42.785 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:12:42.798 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:12:42.806 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:12:42.806 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:12:42.809 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:12:42.811 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:12:42.826 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:12:42.832 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:12:42.832 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:12:42.866 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:12:42.868 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:12:42.893 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:12:42.963 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:12:42.977 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:12:42.977 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T00:12:42.977 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:12:42.978 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:12:42.978 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:12:42.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:42.995 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-25T00:12:42.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:42.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:42.997 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-04-25T00:12:42.997 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-25T00:12:43.053 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:12:43.053 INFO:teuthology.task.kernel:Running kernel on smithi169: 5.14.0-437.el9.x86_64 2024-04-25T00:12:43.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum install -y kernel 2024-04-25T00:12:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:12:43.054 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-437.el9.x86_64 2024-04-25T00:12:43.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-04-25T00:12:43.056 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:12:43.056 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-437.el9.x86_64 2024-04-25T00:12:43.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-04-25T00:12:45.123 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-25T00:12:45.867 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-25T00:12:47.531 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.0 MB 00:02 2024-04-25T00:12:47.760 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T00:12:47.999 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T00:12:52.085 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.2 MB 00:00 2024-04-25T00:12:52.099 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-25T00:12:53.811 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T00:12:54.384 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-25T00:12:55.804 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T00:12:55.865 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T00:13:00.977 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T00:13:01.095 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T00:13:02.899 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T00:13:02.900 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T00:13:02.981 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T00:13:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-25T00:13:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T00:13:03.027 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T00:13:03.027 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T00:13:03.083 DEBUG:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2024-04-25T00:13:03.110 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T00:13:03.110 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-25T00:13:03.110 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T00:13:03.204 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2024-04-25T00:13:03.603 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:13:00 AM UTC. 2024-04-25T00:13:03.722 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 12:13:01 AM UTC. 2024-04-25T00:13:03.727 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T00:13:03.736 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:13:03.737 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-25T00:13:03.737 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-25T00:13:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:Operation aborted. 2024-04-25T00:13:03.796 INFO:teuthology.orchestra.run.smithi040.stdout:Is this ok [y/N]: 2024-04-25T00:13:03.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2024-04-25T00:13:03.845 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T00:13:03.872 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T00:13:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-25T00:13:03.872 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T00:13:03.872 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-25T00:13:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:13:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:13:03.873 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T00:13:03.873 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T00:13:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:13:03.873 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-25T00:13:03.873 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-25T00:13:03.874 INFO:teuthology.orchestra.run.smithi031.stderr:Operation aborted. 2024-04-25T00:13:03.928 INFO:teuthology.orchestra.run.smithi031.stdout:Is this ok [y/N]: 2024-04-25T00:13:03.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2024-04-25T00:13:04.357 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 12:13:00 AM UTC. 2024-04-25T00:13:04.443 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:13:01 AM UTC. 2024-04-25T00:13:04.476 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T00:13:04.476 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T00:13:04.477 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:13:04.478 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-25T00:13:04.478 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-25T00:13:04.478 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-25T00:13:04.563 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T00:13:04.563 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T00:13:04.563 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T00:13:04.564 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:13:04.565 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-25T00:13:04.565 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-25T00:13:04.565 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-25T00:13:04.745 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T00:13:04.745 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:13:04.745 INFO:teuthology.orchestra.run.smithi040.stdout:Total 952 kB/s | 255 kB 00:00 2024-04-25T00:13:04.762 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-25T00:13:04.944 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-25T00:13:04.944 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-25T00:13:04.949 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-25T00:13:04.950 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-25T00:13:05.095 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-25T00:13:05.105 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-25T00:13:05.196 INFO:teuthology.orchestra.run.smithi040.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:13:05.196 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:05.923 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:05.924 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:13:06.661 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:06.661 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:13:06.661 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalled: 2024-04-25T00:13:06.661 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T00:13:06.661 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:13:06.662 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T00:13:06.851 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T00:13:06.939 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T00:13:06.939 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T00:13:06.939 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T00:13:06.939 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T00:13:06.940 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:13:06.974 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-04-25T00:13:06.993 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:08.089 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-25T00:13:08.089 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:13:08.089 INFO:teuthology.orchestra.run.smithi031.stdout:Total 72 kB/s | 255 kB 00:03 2024-04-25T00:13:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-25T00:13:08.314 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-25T00:13:08.314 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-25T00:13:08.319 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-25T00:13:08.320 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-25T00:13:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-25T00:13:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:13:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:13:09.880 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-25T00:13:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:13:09.885 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalled: 2024-04-25T00:13:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T00:13:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:13:09.885 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T00:13:10.060 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T00:13:10.107 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T00:13:10.108 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T00:13:10.108 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T00:13:10.108 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T00:13:10.108 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:13:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-04-25T00:13:10.161 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:14.181 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 5.5 kB/s | 16 kB 00:02 2024-04-25T00:13:21.090 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 21 MB 00:06 2024-04-25T00:13:26.231 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T00:13:28.071 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T00:13:28.071 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T00:13:28.145 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T00:13:28.146 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-04-25T00:13:28.146 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T00:13:28.244 DEBUG:teuthology.orchestra.run.smithi169:> echo no | sudo yum reinstall kernel || true 2024-04-25T00:13:28.741 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 12:13:26 AM UTC. 2024-04-25T00:13:28.861 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T00:13:28.875 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-25T00:13:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-04-25T00:13:28.875 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout:Reinstalling: 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:13:28.876 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 255 k 2024-04-25T00:13:28.877 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 0 2024-04-25T00:13:28.877 INFO:teuthology.orchestra.run.smithi169.stderr:Operation aborted. 2024-04-25T00:13:28.930 INFO:teuthology.orchestra.run.smithi169.stdout:Is this ok [y/N]: 2024-04-25T00:13:28.930 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum reinstall -y kernel || true 2024-04-25T00:13:29.422 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:13:26 AM UTC. 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout:Reinstalling: 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T00:13:29.539 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-25T00:13:29.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:13:29.540 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 255 k 2024-04-25T00:13:29.540 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 0 2024-04-25T00:13:29.540 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-25T00:13:31.350 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-25T00:13:31.350 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:13:31.353 INFO:teuthology.orchestra.run.smithi169.stdout:Total 141 kB/s | 255 kB 00:01 2024-04-25T00:13:31.371 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-25T00:13:31.557 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-25T00:13:31.557 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-25T00:13:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-25T00:13:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-25T00:13:31.700 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-25T00:13:31.776 INFO:teuthology.orchestra.run.smithi169.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:13:31.776 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:32.426 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:32.426 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:13:33.137 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:13:33.138 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:13:33.138 INFO:teuthology.orchestra.run.smithi169.stdout:Reinstalled: 2024-04-25T00:13:33.138 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T00:13:33.138 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:13:33.138 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T00:13:33.295 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T00:13:33.341 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T00:13:33.342 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T00:13:33.342 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T00:13:33.342 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-25T00:13:33.342 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:13:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2024-04-25T00:13:33.398 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:33.398 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:13:33.462 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:13:33.463 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:13:33.465 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:13:33.468 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:13:33.484 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:13:33.495 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:13:33.502 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:13:33.502 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:13:33.524 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:13:33.526 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:13:33.563 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:13:33.571 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:13:33.571 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:13:33.589 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:13:33.591 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:13:33.616 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:13:33.621 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:13:33.626 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:13:33.632 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:13:33.637 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:13:33.660 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:13:33.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:13:33.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:13:33.681 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:13:33.707 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:13:33.723 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:13:33.723 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:13:33.747 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:13:33.749 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:13:33.764 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:13:33.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:13:33.897 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:13:33.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:13:34.035 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:13:34.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:13:34.111 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:13:34.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:13:34.221 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:13:34.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:13:34.355 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:13:34.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:13:34.429 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:13:34.470 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:13:34.523 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:13:34.607 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:13:34.648 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:13:34.648 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:13:34.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-04-25T00:13:34.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-25T00:13:34.719 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-25T00:13:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:13:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:13:34.793 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:13:35.184 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:13:35.384 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:13:35.384 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:13:35.475 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:13:35.511 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T00:13:35.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-04-25T00:13:35.563 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-04-25T00:13:35.564 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-04-25T00:13:35.644 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-04-25T00:13:35.646 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-25T00:13:35.715 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-25T00:13:35.717 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:13:35.717 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-04-25T00:13:35.741 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-04-25T00:13:35.741 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-04-25T00:13:35.761 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-04-25T00:13:35.761 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-25T00:13:35.781 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-25T00:13:35.781 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-25T00:13:35.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:13:35.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:13:35.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:13:35.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:13:35.832 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:13:35.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:13:35.858 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:13:35.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:13:35.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:13:35.911 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:13:35.935 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:13:35.950 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T00:13:35.956 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T00:13:36.112 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:13:36.129 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:13:36.131 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi040.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:17:10.073 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-25T00:17:10.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T00:17:10.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:17:10.152 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-25T00:17:10.235 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T00:17:10.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-25T00:17:10.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:17:10.310 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-25T00:17:10.403 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-25T00:17:10.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T00:17:10.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:17:10.482 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T00:17:10.557 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T00:17:10.558 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:17:10.574 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:17:10.574 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:17:10.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:17:10.577 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:17:10.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:17:10.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:17:10.580 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:17:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:17:10.610 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:17:10.623 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:17:10.625 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:17:10.626 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:17:10.639 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:17:10.648 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-04-25T00:17:10.651 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-04-25T00:17:10.660 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-04-25T00:17:10.662 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-25T00:17:10.662 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-04-25T00:17:10.673 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-25T00:17:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:17:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:17:10.686 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:17:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:17:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:17:10.698 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:17:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:17:10.746 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-25T00:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.750 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:17:10.751 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-25T00:17:10.751 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.751 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.751 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.751 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.752 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:17:10.756 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:17:10.756 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T00:17:10.756 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.756 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.757 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.757 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:17:10.757 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:17:10.766 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:17:10.767 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:17:10.767 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:17:10.767 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:17:10.774 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:17:10.775 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:17:10.775 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:17:10.775 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:17:10.777 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:17:10.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:17:10.778 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:17:10.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:17:10.924 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:17:10.925 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:17:10.932 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:17:10.932 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:17:10.936 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:17:10.936 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:17:11.037 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:17:11.037 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:17:11.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:17:11.040 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:17:11.040 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:17:11.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:17:11.064 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:17:11.064 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:17:11.066 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:17:11.066 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:17:11.101 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:17:11.101 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:17:11.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:17:11.127 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:17:11.128 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:17:11.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:17:11.130 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:17:11.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:17:11.203 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:17:11.204 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:17:11.266 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-25T00:17:11.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-25T00:17:11.268 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-04-25T00:17:11.269 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:17:11.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-04-25T00:17:11.332 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-04-25T00:17:11.334 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-04-25T00:17:11.492 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-25T00:17:11.502 INFO:teuthology.orchestra.run.smithi031.stdout:48 files removed 2024-04-25T00:17:11.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:17:11.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:17:11.567 INFO:teuthology.orchestra.run.smithi040.stdout:48 files removed 2024-04-25T00:17:11.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:17:12.220 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 190 kB/s | 83 kB 00:00 2024-04-25T00:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 182 kB/s | 83 kB 00:00 2024-04-25T00:17:12.385 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 161 kB/s | 83 kB 00:00 2024-04-25T00:17:12.954 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-25T00:17:13.028 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 16 kB/s | 11 kB 00:00 2024-04-25T00:17:13.093 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-25T00:17:13.563 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-04-25T00:17:13.653 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-25T00:17:13.835 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-25T00:17:14.236 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T00:17:14.419 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T00:17:14.471 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-04-25T00:17:16.245 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T00:17:16.279 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-25T00:17:18.848 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 19 MB 00:03 2024-04-25T00:17:20.413 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T00:17:20.471 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T00:17:22.106 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-25T00:17:22.240 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T00:17:22.865 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T00:17:23.592 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T00:17:24.524 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T00:17:28.323 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 21 MB 00:05 2024-04-25T00:17:28.958 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T00:17:31.161 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:17:31.161 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:17:31.173 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:17:31.173 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:17:31.174 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:17:31.176 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:17:31.177 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:17:31.179 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:17:31.180 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:17:31.245 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T00:17:31.254 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T00:17:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-25T00:17:31.254 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:17:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:17:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:17:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:17:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:17:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:17:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:17:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:17:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:17:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:17:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:17:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:17:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:17:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:17:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:17:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout:Install 148 Packages 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 4 Packages 2024-04-25T00:17:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:17:31.269 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 220 M 2024-04-25T00:17:31.269 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-25T00:17:31.483 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 21 MB 00:06 2024-04-25T00:17:32.034 INFO:teuthology.orchestra.run.smithi031.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-04-25T00:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.4 MB/s | 890 kB 00:00 2024-04-25T00:17:32.351 INFO:teuthology.orchestra.run.smithi031.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.4 MB/s | 145 kB 00:00 2024-04-25T00:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 6.0 MB/s | 2.4 MB 00:00 2024-04-25T00:17:32.960 INFO:teuthology.orchestra.run.smithi031.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-04-25T00:17:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-04-25T00:17:33.577 INFO:teuthology.orchestra.run.smithi031.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 7.6 MB/s | 4.7 MB 00:00 2024-04-25T00:17:33.740 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-04-25T00:17:34.469 INFO:teuthology.orchestra.run.smithi031.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.0 MB/s | 21 MB 00:02 2024-04-25T00:17:34.586 INFO:teuthology.orchestra.run.smithi031.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 216 kB/s | 25 kB 00:00 2024-04-25T00:17:35.328 INFO:teuthology.orchestra.run.smithi031.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 5.8 MB/s | 10 MB 00:01 2024-04-25T00:17:35.453 INFO:teuthology.orchestra.run.smithi031.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 222 kB 00:00 2024-04-25T00:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout:(12/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 6.2 MB/s | 17 MB 00:02 2024-04-25T00:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 98 kB/s | 27 kB 00:00 2024-04-25T00:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 346 kB/s | 32 kB 00:00 2024-04-25T00:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-25T00:17:35.924 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:17:35.925 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout:(16/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.6 MB/s | 745 kB 00:00 2024-04-25T00:17:35.936 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:17:35.937 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:17:35.938 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:17:35.939 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:17:35.940 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:17:35.942 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:17:35.943 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.2 MB/s | 127 kB 00:00 2024-04-25T00:17:36.007 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T00:17:36.016 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T00:17:36.016 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:17:36.017 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:17:36.018 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:17:36.019 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:17:36.020 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:17:36.021 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:17:36.022 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:17:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:17:36.024 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:17:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:17:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:17:36.027 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout:Install 148 Packages 2024-04-25T00:17:36.028 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 4 Packages 2024-04-25T00:17:36.029 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:17:36.029 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 220 M 2024-04-25T00:17:36.029 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-25T00:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.5 MB/s | 501 kB 00:00 2024-04-25T00:17:36.305 INFO:teuthology.orchestra.run.smithi031.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 193 kB/s | 45 kB 00:00 2024-04-25T00:17:36.422 INFO:teuthology.orchestra.run.smithi031.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.1 MB/s | 133 kB 00:00 2024-04-25T00:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.1 MB/s | 162 kB 00:00 2024-04-25T00:17:36.681 INFO:teuthology.orchestra.run.smithi031.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 321 kB 00:00 2024-04-25T00:17:36.724 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 5.5 kB/s | 1.7 kB 00:00 2024-04-25T00:17:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 299 kB 00:00 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi031.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 747 kB/s | 99 kB 00:00 2024-04-25T00:17:37.025 INFO:teuthology.orchestra.run.smithi040.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T00:17:37.040 INFO:teuthology.orchestra.run.smithi031.stdout:(25/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-04-25T00:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 680 kB/s | 85 kB 00:00 2024-04-25T00:17:37.199 INFO:teuthology.orchestra.run.smithi031.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-25T00:17:37.307 INFO:teuthology.orchestra.run.smithi031.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 294 kB/s | 32 kB 00:00 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi031.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.0 MB/s | 141 kB 00:00 2024-04-25T00:17:37.758 INFO:teuthology.orchestra.run.smithi031.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.3 MB/s | 3.1 MB 00:00 2024-04-25T00:17:38.208 INFO:teuthology.orchestra.run.smithi040.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 752 kB/s | 890 kB 00:01 2024-04-25T00:17:38.333 INFO:teuthology.orchestra.run.smithi031.stdout:(31/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 13 MB/s | 48 MB 00:03 2024-04-25T00:17:38.416 INFO:teuthology.orchestra.run.smithi040.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 696 kB/s | 145 kB 00:00 2024-04-25T00:17:38.484 INFO:teuthology.orchestra.run.smithi031.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.6 MB/s | 249 kB 00:00 2024-04-25T00:17:38.592 INFO:teuthology.orchestra.run.smithi031.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 456 kB/s | 49 kB 00:00 2024-04-25T00:17:38.709 INFO:teuthology.orchestra.run.smithi031.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 131 kB/s | 15 kB 00:00 2024-04-25T00:17:38.759 INFO:teuthology.orchestra.run.smithi031.stdout:(35/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 2.7 MB/s | 3.5 MB 00:01 2024-04-25T00:17:38.826 INFO:teuthology.orchestra.run.smithi031.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 264 kB 00:00 2024-04-25T00:17:38.842 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:17:38.842 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:17:38.853 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:17:38.854 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:17:38.854 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:17:38.855 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:17:38.857 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:17:38.859 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:17:38.860 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:17:38.910 INFO:teuthology.orchestra.run.smithi031.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 218 kB/s | 18 kB 00:00 2024-04-25T00:17:38.920 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T00:17:38.930 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T00:17:38.930 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-25T00:17:38.930 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T00:17:38.930 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-25T00:17:38.930 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:17:38.931 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:17:38.932 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-25T00:17:38.933 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:17:38.934 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:17:38.935 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:17:38.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:17:38.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:17:38.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:17:38.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:17:38.940 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 4 Packages 2024-04-25T00:17:38.941 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:17:38.942 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 220 M 2024-04-25T00:17:38.942 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-25T00:17:39.019 INFO:teuthology.orchestra.run.smithi031.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 2.9 MB/s | 762 kB 00:00 2024-04-25T00:17:39.033 INFO:teuthology.orchestra.run.smithi040.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-25T00:17:39.184 INFO:teuthology.orchestra.run.smithi040.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 1.5 MB 00:00 2024-04-25T00:17:39.477 INFO:teuthology.orchestra.run.smithi031.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 142 kB/s | 80 kB 00:00 2024-04-25T00:17:39.509 INFO:teuthology.orchestra.run.smithi040.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-25T00:17:39.919 INFO:teuthology.orchestra.run.smithi031.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 442 kB/s | 195 kB 00:00 2024-04-25T00:17:40.011 INFO:teuthology.orchestra.run.smithi031.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 809 kB/s | 803 kB 00:00 2024-04-25T00:17:40.094 INFO:teuthology.orchestra.run.smithi031.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-04-25T00:17:40.234 INFO:teuthology.orchestra.run.smithi040.stdout:(7/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-25T00:17:40.353 INFO:teuthology.orchestra.run.smithi031.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-04-25T00:17:40.418 INFO:teuthology.orchestra.run.smithi040.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 18 MB/s | 17 MB 00:00 2024-04-25T00:17:40.420 INFO:teuthology.orchestra.run.smithi031.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 506 kB/s | 253 kB 00:00 2024-04-25T00:17:40.460 INFO:teuthology.orchestra.run.smithi040.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 609 kB/s | 25 kB 00:00 2024-04-25T00:17:40.495 INFO:teuthology.orchestra.run.smithi031.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 952 kB/s | 135 kB 00:00 2024-04-25T00:17:40.509 INFO:teuthology.orchestra.run.smithi169.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T00:17:40.604 INFO:teuthology.orchestra.run.smithi031.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-25T00:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 598 kB/s | 125 kB 00:00 2024-04-25T00:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout:(48/152): python3-ply-3.11-14.el9.noarch.rpm 190 kB/s | 106 kB 00:00 2024-04-25T00:17:40.730 INFO:teuthology.orchestra.run.smithi031.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-25T00:17:40.880 INFO:teuthology.orchestra.run.smithi031.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 21 kB 00:00 2024-04-25T00:17:40.905 INFO:teuthology.orchestra.run.smithi031.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 426 kB/s | 107 kB 00:00 2024-04-25T00:17:40.989 INFO:teuthology.orchestra.run.smithi031.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-25T00:17:41.014 INFO:teuthology.orchestra.run.smithi031.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-04-25T00:17:41.131 INFO:teuthology.orchestra.run.smithi031.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-04-25T00:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-25T00:17:41.365 INFO:teuthology.orchestra.run.smithi031.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T00:17:41.482 INFO:teuthology.orchestra.run.smithi031.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-04-25T00:17:41.634 INFO:teuthology.orchestra.run.smithi169.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 791 kB/s | 890 kB 00:01 2024-04-25T00:17:41.673 INFO:teuthology.orchestra.run.smithi031.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-25T00:17:41.826 INFO:teuthology.orchestra.run.smithi169.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 756 kB/s | 145 kB 00:00 2024-04-25T00:17:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:(59/152): bison-3.7.4-5.el9.x86_64.rpm 763 kB/s | 943 kB 00:01 2024-04-25T00:17:41.890 INFO:teuthology.orchestra.run.smithi031.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-25T00:17:41.958 INFO:teuthology.orchestra.run.smithi031.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-25T00:17:41.991 INFO:teuthology.orchestra.run.smithi031.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-25T00:17:42.100 INFO:teuthology.orchestra.run.smithi031.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T00:17:42.192 INFO:teuthology.orchestra.run.smithi031.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 384 kB/s | 35 kB 00:00 2024-04-25T00:17:42.309 INFO:teuthology.orchestra.run.smithi031.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 834 kB/s | 292 kB 00:00 2024-04-25T00:17:42.992 INFO:teuthology.orchestra.run.smithi031.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.5 MB/s | 3.0 MB 00:02 2024-04-25T00:17:43.184 INFO:teuthology.orchestra.run.smithi169.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-25T00:17:43.410 INFO:teuthology.orchestra.run.smithi169.stdout:(5/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 6.3 MB/s | 21 MB 00:03 2024-04-25T00:17:43.585 INFO:teuthology.orchestra.run.smithi169.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-25T00:17:43.618 INFO:teuthology.orchestra.run.smithi040.stdout:(10/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 15 MB/s | 48 MB 00:03 2024-04-25T00:17:43.685 INFO:teuthology.orchestra.run.smithi040.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.3 MB/s | 222 kB 00:00 2024-04-25T00:17:43.735 INFO:teuthology.orchestra.run.smithi040.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 542 kB/s | 27 kB 00:00 2024-04-25T00:17:43.761 INFO:teuthology.orchestra.run.smithi169.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-25T00:17:43.777 INFO:teuthology.orchestra.run.smithi040.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 762 kB/s | 32 kB 00:00 2024-04-25T00:17:43.894 INFO:teuthology.orchestra.run.smithi040.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 6.2 MB/s | 745 kB 00:00 2024-04-25T00:17:43.961 INFO:teuthology.orchestra.run.smithi040.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-25T00:17:44.028 INFO:teuthology.orchestra.run.smithi040.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-25T00:17:44.111 INFO:teuthology.orchestra.run.smithi040.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.9 MB/s | 501 kB 00:00 2024-04-25T00:17:44.667 INFO:teuthology.orchestra.run.smithi031.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.9 MB/s | 4.7 MB 00:02 2024-04-25T00:17:44.678 INFO:teuthology.orchestra.run.smithi040.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 9.4 MB/s | 5.3 MB 00:00 2024-04-25T00:17:44.754 INFO:teuthology.orchestra.run.smithi040.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 604 kB/s | 45 kB 00:00 2024-04-25T00:17:44.804 INFO:teuthology.orchestra.run.smithi040.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.6 MB/s | 133 kB 00:00 2024-04-25T00:17:44.854 INFO:teuthology.orchestra.run.smithi040.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 3.2 MB/s | 162 kB 00:00 2024-04-25T00:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 9.2 MB/s | 10 MB 00:01 2024-04-25T00:17:44.913 INFO:teuthology.orchestra.run.smithi040.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 5.4 MB/s | 321 kB 00:00 2024-04-25T00:17:44.936 INFO:teuthology.orchestra.run.smithi169.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 378 kB/s | 25 kB 00:00 2024-04-25T00:17:44.942 INFO:teuthology.orchestra.run.smithi031.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-25T00:17:44.971 INFO:teuthology.orchestra.run.smithi040.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 5.0 MB/s | 299 kB 00:00 2024-04-25T00:17:45.043 INFO:teuthology.orchestra.run.smithi031.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-25T00:17:45.063 INFO:teuthology.orchestra.run.smithi040.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T00:17:45.147 INFO:teuthology.orchestra.run.smithi040.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T00:17:45.189 INFO:teuthology.orchestra.run.smithi031.stdout:(70/152): postgresql-13.14-1.el9.x86_64.rpm 740 kB/s | 1.6 MB 00:02 2024-04-25T00:17:45.330 INFO:teuthology.orchestra.run.smithi169.stdout:(10/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x 1.0 MB/s | 5.4 MB 00:05 2024-04-25T00:17:45.356 INFO:teuthology.orchestra.run.smithi040.stdout:(26/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.4 MB/s | 21 MB 00:08 2024-04-25T00:17:45.536 INFO:teuthology.orchestra.run.smithi169.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.0 MB/s | 222 kB 00:00 2024-04-25T00:17:45.556 INFO:teuthology.orchestra.run.smithi040.stdout:(27/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.6 MB/s | 3.1 MB 00:00 2024-04-25T00:17:45.582 INFO:teuthology.orchestra.run.smithi040.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 759 kB/s | 171 kB 00:00 2024-04-25T00:17:45.607 INFO:teuthology.orchestra.run.smithi040.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 630 kB/s | 32 kB 00:00 2024-04-25T00:17:45.691 INFO:teuthology.orchestra.run.smithi040.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.3 MB/s | 141 kB 00:00 2024-04-25T00:17:45.695 INFO:teuthology.orchestra.run.smithi169.stdout:(12/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 8.0 MB/s | 17 MB 00:02 2024-04-25T00:17:45.746 INFO:teuthology.orchestra.run.smithi169.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 129 kB/s | 27 kB 00:00 2024-04-25T00:17:45.787 INFO:teuthology.orchestra.run.smithi169.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 346 kB/s | 32 kB 00:00 2024-04-25T00:17:45.871 INFO:teuthology.orchestra.run.smithi169.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.9 MB/s | 163 kB 00:00 2024-04-25T00:17:45.900 INFO:teuthology.orchestra.run.smithi040.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 12 MB/s | 3.5 MB 00:00 2024-04-25T00:17:45.946 INFO:teuthology.orchestra.run.smithi169.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-25T00:17:45.958 INFO:teuthology.orchestra.run.smithi040.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 4.2 MB/s | 249 kB 00:00 2024-04-25T00:17:45.959 INFO:teuthology.orchestra.run.smithi031.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 705 kB/s | 911 kB 00:01 2024-04-25T00:17:46.008 INFO:teuthology.orchestra.run.smithi040.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 992 kB/s | 49 kB 00:00 2024-04-25T00:17:46.042 INFO:teuthology.orchestra.run.smithi040.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 459 kB/s | 15 kB 00:00 2024-04-25T00:17:46.046 INFO:teuthology.orchestra.run.smithi169.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.9 MB/s | 501 kB 00:00 2024-04-25T00:17:46.101 INFO:teuthology.orchestra.run.smithi040.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 4.4 MB/s | 264 kB 00:00 2024-04-25T00:17:46.143 INFO:teuthology.orchestra.run.smithi031.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-25T00:17:46.180 INFO:teuthology.orchestra.run.smithi169.stdout:(18/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 1.7 MB/s | 745 kB 00:00 2024-04-25T00:17:46.234 INFO:teuthology.orchestra.run.smithi040.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T00:17:46.264 INFO:teuthology.orchestra.run.smithi169.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 541 kB/s | 45 kB 00:00 2024-04-25T00:17:46.293 INFO:teuthology.orchestra.run.smithi040.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 314 kB/s | 18 kB 00:00 2024-04-25T00:17:46.402 INFO:teuthology.orchestra.run.smithi040.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T00:17:46.422 INFO:teuthology.orchestra.run.smithi169.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 840 kB/s | 133 kB 00:00 2024-04-25T00:17:46.502 INFO:teuthology.orchestra.run.smithi040.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-04-25T00:17:46.544 INFO:teuthology.orchestra.run.smithi040.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T00:17:46.573 INFO:teuthology.orchestra.run.smithi169.stdout:(21/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 5.3 MB 00:00 2024-04-25T00:17:46.586 INFO:teuthology.orchestra.run.smithi040.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T00:17:46.593 INFO:teuthology.orchestra.run.smithi031.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 736 kB/s | 1.0 MB 00:01 2024-04-25T00:17:46.598 INFO:teuthology.orchestra.run.smithi169.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 925 kB/s | 162 kB 00:00 2024-04-25T00:17:46.653 INFO:teuthology.orchestra.run.smithi040.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T00:17:46.682 INFO:teuthology.orchestra.run.smithi169.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 321 kB 00:00 2024-04-25T00:17:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T00:17:46.737 INFO:teuthology.orchestra.run.smithi040.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T00:17:46.748 INFO:teuthology.orchestra.run.smithi169.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 99 kB 00:00 2024-04-25T00:17:46.779 INFO:teuthology.orchestra.run.smithi040.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T00:17:46.782 INFO:teuthology.orchestra.run.smithi169.stdout:(25/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.6 MB/s | 299 kB 00:00 2024-04-25T00:17:46.816 INFO:teuthology.orchestra.run.smithi169.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-25T00:17:46.820 INFO:teuthology.orchestra.run.smithi040.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T00:17:46.827 INFO:teuthology.orchestra.run.smithi031.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 3.2 MB/s | 5.8 MB 00:01 2024-04-25T00:17:46.883 INFO:teuthology.orchestra.run.smithi169.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-25T00:17:46.941 INFO:teuthology.orchestra.run.smithi169.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 544 kB/s | 32 kB 00:00 2024-04-25T00:17:46.944 INFO:teuthology.orchestra.run.smithi031.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-25T00:17:46.977 INFO:teuthology.orchestra.run.smithi031.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 636 kB/s | 244 kB 00:00 2024-04-25T00:17:47.008 INFO:teuthology.orchestra.run.smithi169.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.1 MB/s | 141 kB 00:00 2024-04-25T00:17:47.061 INFO:teuthology.orchestra.run.smithi031.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-04-25T00:17:47.086 INFO:teuthology.orchestra.run.smithi031.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-25T00:17:47.178 INFO:teuthology.orchestra.run.smithi031.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-25T00:17:47.211 INFO:teuthology.orchestra.run.smithi031.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 278 kB/s | 35 kB 00:00 2024-04-25T00:17:47.221 INFO:teuthology.orchestra.run.smithi040.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2024-04-25T00:17:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.4 MB/s | 3.5 MB 00:00 2024-04-25T00:17:47.504 INFO:teuthology.orchestra.run.smithi040.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-25T00:17:47.563 INFO:teuthology.orchestra.run.smithi040.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T00:17:47.637 INFO:teuthology.orchestra.run.smithi031.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 589 kB/s | 250 kB 00:00 2024-04-25T00:17:47.638 INFO:teuthology.orchestra.run.smithi040.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 287 kB/s | 21 kB 00:00 2024-04-25T00:17:47.814 INFO:teuthology.orchestra.run.smithi040.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-25T00:17:47.896 INFO:teuthology.orchestra.run.smithi031.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 859 kB/s | 222 kB 00:00 2024-04-25T00:17:47.914 INFO:teuthology.orchestra.run.smithi040.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 33 kB 00:00 2024-04-25T00:17:48.013 INFO:teuthology.orchestra.run.smithi031.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 662 kB/s | 77 kB 00:00 2024-04-25T00:17:48.163 INFO:teuthology.orchestra.run.smithi031.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 282 kB/s | 42 kB 00:00 2024-04-25T00:17:48.369 INFO:teuthology.orchestra.run.smithi169.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-04-25T00:17:48.478 INFO:teuthology.orchestra.run.smithi169.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.2 MB/s | 249 kB 00:00 2024-04-25T00:17:48.570 INFO:teuthology.orchestra.run.smithi169.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 536 kB/s | 49 kB 00:00 2024-04-25T00:17:48.639 INFO:teuthology.orchestra.run.smithi040.stdout:(53/152): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-04-25T00:17:48.646 INFO:teuthology.orchestra.run.smithi169.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 204 kB/s | 15 kB 00:00 2024-04-25T00:17:48.671 INFO:teuthology.orchestra.run.smithi031.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 530 kB/s | 269 kB 00:00 2024-04-25T00:17:48.805 INFO:teuthology.orchestra.run.smithi031.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T00:17:48.831 INFO:teuthology.orchestra.run.smithi040.stdout:(54/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 1.2 MB/s | 10 MB 00:08 2024-04-25T00:17:48.858 INFO:teuthology.orchestra.run.smithi040.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 84 kB/s | 18 kB 00:00 2024-04-25T00:17:48.915 INFO:teuthology.orchestra.run.smithi040.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:01 2024-04-25T00:17:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-25T00:17:48.990 INFO:teuthology.orchestra.run.smithi040.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 70 kB/s | 11 kB 00:00 2024-04-25T00:17:49.016 INFO:teuthology.orchestra.run.smithi040.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-25T00:17:49.055 INFO:teuthology.orchestra.run.smithi031.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-04-25T00:17:49.083 INFO:teuthology.orchestra.run.smithi040.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 709 kB/s | 160 kB 00:00 2024-04-25T00:17:49.138 INFO:teuthology.orchestra.run.smithi169.stdout:(35/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 11 MB/s | 48 MB 00:04 2024-04-25T00:17:49.141 INFO:teuthology.orchestra.run.smithi040.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-04-25T00:17:49.163 INFO:teuthology.orchestra.run.smithi169.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 510 kB/s | 264 kB 00:00 2024-04-25T00:17:49.164 INFO:teuthology.orchestra.run.smithi031.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-04-25T00:17:49.200 INFO:teuthology.orchestra.run.smithi040.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-25T00:17:49.225 INFO:teuthology.orchestra.run.smithi040.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-04-25T00:17:49.230 INFO:teuthology.orchestra.run.smithi169.stdout:(37/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-25T00:17:49.259 INFO:teuthology.orchestra.run.smithi040.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-25T00:17:49.280 INFO:teuthology.orchestra.run.smithi169.stdout:(38/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 1.2 MB/s | 3.1 MB 00:02 2024-04-25T00:17:49.306 INFO:teuthology.orchestra.run.smithi169.stdout:(39/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 128 kB/s | 18 kB 00:00 2024-04-25T00:17:49.359 INFO:teuthology.orchestra.run.smithi040.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-04-25T00:17:49.384 INFO:teuthology.orchestra.run.smithi040.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 281 kB/s | 35 kB 00:00 2024-04-25T00:17:49.409 INFO:teuthology.orchestra.run.smithi040.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-04-25T00:17:49.531 INFO:teuthology.orchestra.run.smithi169.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 267 kB/s | 80 kB 00:00 2024-04-25T00:17:49.656 INFO:teuthology.orchestra.run.smithi169.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 558 kB/s | 195 kB 00:00 2024-04-25T00:17:49.660 INFO:teuthology.orchestra.run.smithi040.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.4 MB/s | 1.6 MB 00:00 2024-04-25T00:17:49.706 INFO:teuthology.orchestra.run.smithi169.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-04-25T00:17:49.765 INFO:teuthology.orchestra.run.smithi169.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T00:17:49.785 INFO:teuthology.orchestra.run.smithi040.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-25T00:17:49.825 INFO:teuthology.orchestra.run.smithi169.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-25T00:17:49.849 INFO:teuthology.orchestra.run.smithi169.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-25T00:17:49.899 INFO:teuthology.orchestra.run.smithi169.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T00:17:49.920 INFO:teuthology.orchestra.run.smithi040.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 993 kB/s | 133 kB 00:00 2024-04-25T00:17:49.949 INFO:teuthology.orchestra.run.smithi169.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T00:17:50.058 INFO:teuthology.orchestra.run.smithi169.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-25T00:17:50.091 INFO:teuthology.orchestra.run.smithi169.stdout:(49/152): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-04-25T00:17:50.117 INFO:teuthology.orchestra.run.smithi169.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T00:17:50.167 INFO:teuthology.orchestra.run.smithi169.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-04-25T00:17:50.192 INFO:teuthology.orchestra.run.smithi169.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-25T00:17:50.217 INFO:teuthology.orchestra.run.smithi169.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-25T00:17:50.244 INFO:teuthology.orchestra.run.smithi040.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.4 MB/s | 4.7 MB 00:00 2024-04-25T00:17:50.251 INFO:teuthology.orchestra.run.smithi169.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-25T00:17:50.276 INFO:teuthology.orchestra.run.smithi169.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T00:17:50.302 INFO:teuthology.orchestra.run.smithi169.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-25T00:17:50.327 INFO:teuthology.orchestra.run.smithi169.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T00:17:50.352 INFO:teuthology.orchestra.run.smithi169.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T00:17:50.394 INFO:teuthology.orchestra.run.smithi169.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T00:17:50.411 INFO:teuthology.orchestra.run.smithi040.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-25T00:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-04-25T00:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-25T00:17:50.469 INFO:teuthology.orchestra.run.smithi040.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T00:17:50.479 INFO:teuthology.orchestra.run.smithi169.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T00:17:50.504 INFO:teuthology.orchestra.run.smithi169.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T00:17:50.529 INFO:teuthology.orchestra.run.smithi169.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T00:17:50.545 INFO:teuthology.orchestra.run.smithi040.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 9.3 MB/s | 5.8 MB 00:00 2024-04-25T00:17:50.554 INFO:teuthology.orchestra.run.smithi169.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-25T00:17:50.603 INFO:teuthology.orchestra.run.smithi040.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-25T00:17:50.604 INFO:teuthology.orchestra.run.smithi169.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-04-25T00:17:50.662 INFO:teuthology.orchestra.run.smithi040.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-25T00:17:50.688 INFO:teuthology.orchestra.run.smithi169.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-25T00:17:50.720 INFO:teuthology.orchestra.run.smithi040.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-25T00:17:50.730 INFO:teuthology.orchestra.run.smithi169.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T00:17:50.779 INFO:teuthology.orchestra.run.smithi040.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-04-25T00:17:50.797 INFO:teuthology.orchestra.run.smithi169.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T00:17:50.822 INFO:teuthology.orchestra.run.smithi169.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T00:17:50.837 INFO:teuthology.orchestra.run.smithi040.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T00:17:50.856 INFO:teuthology.orchestra.run.smithi169.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-25T00:17:50.898 INFO:teuthology.orchestra.run.smithi169.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-25T00:17:50.923 INFO:teuthology.orchestra.run.smithi169.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T00:17:50.965 INFO:teuthology.orchestra.run.smithi169.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-25T00:17:50.989 INFO:teuthology.orchestra.run.smithi031.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.3 MB/s | 5.1 MB 00:03 2024-04-25T00:17:51.038 INFO:teuthology.orchestra.run.smithi040.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 174 kB/s | 35 kB 00:00 2024-04-25T00:17:51.049 INFO:teuthology.orchestra.run.smithi169.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-25T00:17:51.074 INFO:teuthology.orchestra.run.smithi169.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-25T00:17:51.089 INFO:teuthology.orchestra.run.smithi031.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-04-25T00:17:51.096 INFO:teuthology.orchestra.run.smithi040.stdout:(80/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-25T00:17:51.099 INFO:teuthology.orchestra.run.smithi169.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T00:17:51.125 INFO:teuthology.orchestra.run.smithi169.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T00:17:51.150 INFO:teuthology.orchestra.run.smithi169.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T00:17:51.175 INFO:teuthology.orchestra.run.smithi169.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-04-25T00:17:51.189 INFO:teuthology.orchestra.run.smithi040.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T00:17:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-04-25T00:17:51.247 INFO:teuthology.orchestra.run.smithi040.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T00:17:51.259 INFO:teuthology.orchestra.run.smithi169.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-25T00:17:51.284 INFO:teuthology.orchestra.run.smithi169.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-25T00:17:51.306 INFO:teuthology.orchestra.run.smithi040.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T00:17:51.309 INFO:teuthology.orchestra.run.smithi169.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-25T00:17:51.335 INFO:teuthology.orchestra.run.smithi169.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T00:17:51.348 INFO:teuthology.orchestra.run.smithi031.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-04-25T00:17:51.351 INFO:teuthology.orchestra.run.smithi169.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T00:17:51.364 INFO:teuthology.orchestra.run.smithi040.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-04-25T00:17:51.377 INFO:teuthology.orchestra.run.smithi169.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-25T00:17:51.402 INFO:teuthology.orchestra.run.smithi169.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T00:17:51.423 INFO:teuthology.orchestra.run.smithi040.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T00:17:51.427 INFO:teuthology.orchestra.run.smithi169.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T00:17:51.448 INFO:teuthology.orchestra.run.smithi031.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 360 kB/s | 36 kB 00:00 2024-04-25T00:17:51.461 INFO:teuthology.orchestra.run.smithi169.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 867 kB/s | 51 kB 00:00 2024-04-25T00:17:51.486 INFO:teuthology.orchestra.run.smithi169.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-04-25T00:17:51.532 INFO:teuthology.orchestra.run.smithi040.stdout:(86/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.6 MB/s | 6.0 MB 00:01 2024-04-25T00:17:51.549 INFO:teuthology.orchestra.run.smithi031.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-25T00:17:51.557 INFO:teuthology.orchestra.run.smithi040.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T00:17:51.578 INFO:teuthology.orchestra.run.smithi169.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-25T00:17:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T00:17:51.603 INFO:teuthology.orchestra.run.smithi169.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-04-25T00:17:51.624 INFO:teuthology.orchestra.run.smithi040.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 763 kB/s | 51 kB 00:00 2024-04-25T00:17:51.629 INFO:teuthology.orchestra.run.smithi169.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-04-25T00:17:51.654 INFO:teuthology.orchestra.run.smithi169.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T00:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-25T00:17:51.658 INFO:teuthology.orchestra.run.smithi040.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-04-25T00:17:51.680 INFO:teuthology.orchestra.run.smithi169.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-25T00:17:51.705 INFO:teuthology.orchestra.run.smithi169.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-25T00:17:51.717 INFO:teuthology.orchestra.run.smithi040.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 718 kB/s | 42 kB 00:00 2024-04-25T00:17:51.757 INFO:teuthology.orchestra.run.smithi031.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-04-25T00:17:51.772 INFO:teuthology.orchestra.run.smithi169.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-25T00:17:51.775 INFO:teuthology.orchestra.run.smithi040.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-04-25T00:17:51.797 INFO:teuthology.orchestra.run.smithi169.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-04-25T00:17:51.822 INFO:teuthology.orchestra.run.smithi169.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T00:17:51.842 INFO:teuthology.orchestra.run.smithi040.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-25T00:17:51.847 INFO:teuthology.orchestra.run.smithi169.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T00:17:51.858 INFO:teuthology.orchestra.run.smithi031.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 937 kB/s | 94 kB 00:00 2024-04-25T00:17:51.873 INFO:teuthology.orchestra.run.smithi169.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-25T00:17:51.901 INFO:teuthology.orchestra.run.smithi040.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-25T00:17:51.958 INFO:teuthology.orchestra.run.smithi031.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-04-25T00:17:51.959 INFO:teuthology.orchestra.run.smithi040.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-04-25T00:17:52.058 INFO:teuthology.orchestra.run.smithi031.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-04-25T00:17:52.077 INFO:teuthology.orchestra.run.smithi040.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-25T00:17:52.098 INFO:teuthology.orchestra.run.smithi169.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-25T00:17:52.132 INFO:teuthology.orchestra.run.smithi169.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-25T00:17:52.135 INFO:teuthology.orchestra.run.smithi040.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-04-25T00:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-25T00:17:52.194 INFO:teuthology.orchestra.run.smithi040.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T00:17:52.252 INFO:teuthology.orchestra.run.smithi040.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T00:17:52.293 INFO:teuthology.orchestra.run.smithi031.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-25T00:17:52.311 INFO:teuthology.orchestra.run.smithi040.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 769 kB/s | 45 kB 00:00 2024-04-25T00:17:52.332 INFO:teuthology.orchestra.run.smithi169.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-25T00:17:52.569 INFO:teuthology.orchestra.run.smithi040.stdout:(101/152): python3-numpy-1.20.1-5.el9.x86_64.rp 3.3 MB/s | 5.1 MB 00:01 2024-04-25T00:17:52.591 INFO:teuthology.orchestra.run.smithi169.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T00:17:52.595 INFO:teuthology.orchestra.run.smithi040.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-25T00:17:52.674 INFO:teuthology.orchestra.run.smithi169.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 570 kB/s | 309 kB 00:00 2024-04-25T00:17:52.758 INFO:teuthology.orchestra.run.smithi169.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 832 kB/s | 548 kB 00:00 2024-04-25T00:17:52.783 INFO:teuthology.orchestra.run.smithi169.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T00:17:52.858 INFO:teuthology.orchestra.run.smithi169.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-25T00:17:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 927 kB/s | 548 kB 00:00 2024-04-25T00:17:52.892 INFO:teuthology.orchestra.run.smithi169.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-25T00:17:53.042 INFO:teuthology.orchestra.run.smithi169.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 822 kB/s | 151 kB 00:00 2024-04-25T00:17:53.129 INFO:teuthology.orchestra.run.smithi040.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-25T00:17:53.159 INFO:teuthology.orchestra.run.smithi169.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-25T00:17:53.209 INFO:teuthology.orchestra.run.smithi169.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T00:17:53.260 INFO:teuthology.orchestra.run.smithi169.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-04-25T00:17:53.310 INFO:teuthology.orchestra.run.smithi169.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-25T00:17:53.329 INFO:teuthology.orchestra.run.smithi040.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-25T00:17:53.394 INFO:teuthology.orchestra.run.smithi169.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-04-25T00:17:53.409 INFO:teuthology.orchestra.run.smithi031.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-04-25T00:17:53.419 INFO:teuthology.orchestra.run.smithi169.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2024-04-25T00:17:53.444 INFO:teuthology.orchestra.run.smithi169.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T00:17:53.520 INFO:teuthology.orchestra.run.smithi169.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T00:17:53.545 INFO:teuthology.orchestra.run.smithi169.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-25T00:17:53.570 INFO:teuthology.orchestra.run.smithi169.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-25T00:17:53.576 INFO:teuthology.orchestra.run.smithi031.stdout:(105/152): python3-babel-2.9.1-2.el9.noarch.rpm 821 kB/s | 6.0 MB 00:07 2024-04-25T00:17:53.629 INFO:teuthology.orchestra.run.smithi169.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-25T00:17:53.668 INFO:teuthology.orchestra.run.smithi031.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T00:17:53.671 INFO:teuthology.orchestra.run.smithi169.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T00:17:53.721 INFO:teuthology.orchestra.run.smithi169.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T00:17:53.771 INFO:teuthology.orchestra.run.smithi169.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T00:17:53.787 INFO:teuthology.orchestra.run.smithi040.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 460 kB/s | 548 kB 00:01 2024-04-25T00:17:53.812 INFO:teuthology.orchestra.run.smithi040.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 40 kB/s | 19 kB 00:00 2024-04-25T00:17:53.888 INFO:teuthology.orchestra.run.smithi169.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-04-25T00:17:53.905 INFO:teuthology.orchestra.run.smithi169.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 126 kB/s | 23 kB 00:00 2024-04-25T00:17:53.912 INFO:teuthology.orchestra.run.smithi040.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-25T00:17:53.922 INFO:teuthology.orchestra.run.smithi169.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 129 kB/s | 19 kB 00:00 2024-04-25T00:17:53.935 INFO:teuthology.orchestra.run.smithi031.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-25T00:17:53.981 INFO:teuthology.orchestra.run.smithi169.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-25T00:17:54.014 INFO:teuthology.orchestra.run.smithi169.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-04-25T00:17:54.021 INFO:teuthology.orchestra.run.smithi040.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 346 kB/s | 309 kB 00:00 2024-04-25T00:17:54.081 INFO:teuthology.orchestra.run.smithi169.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-04-25T00:17:54.089 INFO:teuthology.orchestra.run.smithi040.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-04-25T00:17:54.115 INFO:teuthology.orchestra.run.smithi169.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-04-25T00:17:54.140 INFO:teuthology.orchestra.run.smithi169.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2024-04-25T00:17:54.190 INFO:teuthology.orchestra.run.smithi169.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T00:17:54.205 INFO:teuthology.orchestra.run.smithi040.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-25T00:17:54.215 INFO:teuthology.orchestra.run.smithi169.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T00:17:54.241 INFO:teuthology.orchestra.run.smithi169.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-25T00:17:54.277 INFO:teuthology.orchestra.run.smithi031.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T00:17:54.291 INFO:teuthology.orchestra.run.smithi169.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T00:17:54.316 INFO:teuthology.orchestra.run.smithi169.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T00:17:54.350 INFO:teuthology.orchestra.run.smithi169.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T00:17:54.355 INFO:teuthology.orchestra.run.smithi040.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-04-25T00:17:54.392 INFO:teuthology.orchestra.run.smithi169.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-25T00:17:54.417 INFO:teuthology.orchestra.run.smithi169.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-25T00:17:54.451 INFO:teuthology.orchestra.run.smithi169.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-25T00:17:54.460 INFO:teuthology.orchestra.run.smithi031.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-04-25T00:17:54.492 INFO:teuthology.orchestra.run.smithi169.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T00:17:54.526 INFO:teuthology.orchestra.run.smithi169.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-04-25T00:17:54.560 INFO:teuthology.orchestra.run.smithi169.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T00:17:54.593 INFO:teuthology.orchestra.run.smithi169.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T00:17:54.635 INFO:teuthology.orchestra.run.smithi169.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T00:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 188 kB/s | 67 kB 00:00 2024-04-25T00:17:54.702 INFO:teuthology.orchestra.run.smithi169.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T00:17:54.886 INFO:teuthology.orchestra.run.smithi031.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-25T00:17:55.114 INFO:teuthology.orchestra.run.smithi040.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 723 kB/s | 548 kB 00:00 2024-04-25T00:17:55.170 INFO:teuthology.orchestra.run.smithi169.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-04-25T00:17:55.214 INFO:teuthology.orchestra.run.smithi040.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-04-25T00:17:55.300 INFO:teuthology.orchestra.run.smithi040.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 766 kB/s | 838 kB 00:01 2024-04-25T00:17:55.303 INFO:teuthology.orchestra.run.smithi031.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T00:17:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 291 kB/s | 32 kB 00:00 2024-04-25T00:17:55.400 INFO:teuthology.orchestra.run.smithi040.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T00:17:55.512 INFO:teuthology.orchestra.run.smithi031.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 876 kB/s | 548 kB 00:00 2024-04-25T00:17:55.520 INFO:teuthology.orchestra.run.smithi169.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-04-25T00:17:55.578 INFO:teuthology.orchestra.run.smithi169.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.4 MB/s | 3.4 MB 00:00 2024-04-25T00:17:55.600 INFO:teuthology.orchestra.run.smithi040.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 624 kB/s | 172 kB 00:00 2024-04-25T00:17:55.662 INFO:teuthology.orchestra.run.smithi031.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 121 kB/s | 43 kB 00:00 2024-04-25T00:17:55.753 INFO:teuthology.orchestra.run.smithi169.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T00:17:55.755 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:17:55.756 INFO:teuthology.orchestra.run.smithi169.stdout:Total 13 MB/s | 220 MB 00:16 2024-04-25T00:17:55.787 INFO:teuthology.orchestra.run.smithi031.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 117 kB/s | 32 kB 00:00 2024-04-25T00:17:55.901 INFO:teuthology.orchestra.run.smithi040.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 716 kB/s | 358 kB 00:00 2024-04-25T00:17:55.929 INFO:teuthology.orchestra.run.smithi031.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-25T00:17:56.043 INFO:teuthology.orchestra.run.smithi040.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 249 kB/s | 35 kB 00:00 2024-04-25T00:17:56.068 INFO:teuthology.orchestra.run.smithi040.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 648 kB/s | 303 kB 00:00 2024-04-25T00:17:56.221 INFO:teuthology.orchestra.run.smithi031.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-04-25T00:17:56.385 INFO:teuthology.orchestra.run.smithi040.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 602 kB/s | 206 kB 00:00 2024-04-25T00:17:56.497 INFO:teuthology.orchestra.run.smithi031.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 632 kB/s | 358 kB 00:00 2024-04-25T00:17:56.652 INFO:teuthology.orchestra.run.smithi040.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-04-25T00:17:56.761 INFO:teuthology.orchestra.run.smithi040.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-25T00:17:56.822 INFO:teuthology.orchestra.run.smithi031.stdout:(119/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-04-25T00:17:56.847 INFO:teuthology.orchestra.run.smithi031.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 101 kB/s | 35 kB 00:00 2024-04-25T00:17:56.869 INFO:teuthology.orchestra.run.smithi040.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T00:17:56.870 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-25T00:17:56.872 INFO:teuthology.orchestra.run.smithi031.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 466 kB/s | 303 kB 00:00 2024-04-25T00:17:56.970 INFO:teuthology.orchestra.run.smithi040.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T00:17:57.018 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-25T00:17:57.018 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-25T00:17:57.070 INFO:teuthology.orchestra.run.smithi040.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T00:17:57.170 INFO:teuthology.orchestra.run.smithi040.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T00:17:57.305 INFO:teuthology.orchestra.run.smithi031.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-25T00:17:57.331 INFO:teuthology.orchestra.run.smithi031.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 404 kB/s | 206 kB 00:00 2024-04-25T00:17:57.498 INFO:teuthology.orchestra.run.smithi031.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-25T00:17:57.648 INFO:teuthology.orchestra.run.smithi031.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 56 kB/s | 18 kB 00:00 2024-04-25T00:17:57.698 INFO:teuthology.orchestra.run.smithi031.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-04-25T00:17:57.832 INFO:teuthology.orchestra.run.smithi031.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 69 kB/s | 23 kB 00:00 2024-04-25T00:17:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 50 kB/s | 19 kB 00:00 2024-04-25T00:17:58.066 INFO:teuthology.orchestra.run.smithi031.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 54 kB/s | 20 kB 00:00 2024-04-25T00:17:58.391 INFO:teuthology.orchestra.run.smithi031.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-04-25T00:17:58.416 INFO:teuthology.orchestra.run.smithi031.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-04-25T00:17:58.479 INFO:teuthology.orchestra.run.smithi040.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 799 kB/s | 1.0 MB 00:01 2024-04-25T00:17:58.521 INFO:teuthology.orchestra.run.smithi040.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 851 kB/s | 2.0 MB 00:02 2024-04-25T00:17:58.579 INFO:teuthology.orchestra.run.smithi040.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-25T00:17:58.633 INFO:teuthology.orchestra.run.smithi031.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-04-25T00:17:58.705 INFO:teuthology.orchestra.run.smithi040.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2024-04-25T00:17:58.763 INFO:teuthology.orchestra.run.smithi040.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 314 kB/s | 58 kB 00:00 2024-04-25T00:17:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 150 kB/s | 58 kB 00:00 2024-04-25T00:17:58.798 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-25T00:17:58.799 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-25T00:17:58.863 INFO:teuthology.orchestra.run.smithi040.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-25T00:17:58.933 INFO:teuthology.orchestra.run.smithi031.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 527 kB/s | 272 kB 00:00 2024-04-25T00:17:58.958 INFO:teuthology.orchestra.run.smithi031.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 50 kB/s | 16 kB 00:00 2024-04-25T00:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout:(134/152): libarrow-9.0.0-10.el9.x86_64.rpm 859 kB/s | 4.4 MB 00:05 2024-04-25T00:17:59.081 INFO:teuthology.orchestra.run.smithi040.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 415 kB/s | 90 kB 00:00 2024-04-25T00:17:59.117 INFO:teuthology.orchestra.run.smithi031.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 263 kB/s | 90 kB 00:00 2024-04-25T00:17:59.156 INFO:teuthology.orchestra.run.smithi040.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 604 kB/s | 272 kB 00:00 2024-04-25T00:17:59.181 INFO:teuthology.orchestra.run.smithi040.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-04-25T00:17:59.206 INFO:teuthology.orchestra.run.smithi040.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-04-25T00:17:59.209 INFO:teuthology.orchestra.run.smithi031.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-04-25T00:17:59.235 INFO:teuthology.orchestra.run.smithi031.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 112 kB/s | 31 kB 00:00 2024-04-25T00:17:59.315 INFO:teuthology.orchestra.run.smithi040.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2024-04-25T00:17:59.365 INFO:teuthology.orchestra.run.smithi040.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-25T00:17:59.424 INFO:teuthology.orchestra.run.smithi040.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 704 kB/s | 188 kB 00:00 2024-04-25T00:17:59.449 INFO:teuthology.orchestra.run.smithi040.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-04-25T00:17:59.509 INFO:teuthology.orchestra.run.smithi031.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-25T00:17:59.551 INFO:teuthology.orchestra.run.smithi031.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-04-25T00:17:59.577 INFO:teuthology.orchestra.run.smithi031.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 410 kB/s | 188 kB 00:00 2024-04-25T00:17:59.616 INFO:teuthology.orchestra.run.smithi040.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 467 kB/s | 90 kB 00:00 2024-04-25T00:17:59.725 INFO:teuthology.orchestra.run.smithi040.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 639 kB/s | 230 kB 00:00 2024-04-25T00:17:59.759 INFO:teuthology.orchestra.run.smithi040.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-04-25T00:17:59.852 INFO:teuthology.orchestra.run.smithi031.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-25T00:17:59.992 INFO:teuthology.orchestra.run.smithi040.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 718 kB/s | 192 kB 00:00 2024-04-25T00:18:00.002 INFO:teuthology.orchestra.run.smithi031.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 211 kB/s | 90 kB 00:00 2024-04-25T00:18:00.101 INFO:teuthology.orchestra.run.smithi040.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 656 kB/s | 427 kB 00:00 2024-04-25T00:18:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 362 kB/s | 230 kB 00:00 2024-04-25T00:18:00.361 INFO:teuthology.orchestra.run.smithi031.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 56 kB/s | 20 kB 00:00 2024-04-25T00:18:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 648 kB/s | 427 kB 00:00 2024-04-25T00:18:00.734 INFO:teuthology.orchestra.run.smithi040.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-04-25T00:18:00.736 INFO:teuthology.orchestra.run.smithi031.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 348 kB/s | 192 kB 00:00 2024-04-25T00:18:00.893 INFO:teuthology.orchestra.run.smithi040.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-04-25T00:18:00.976 INFO:teuthology.orchestra.run.smithi040.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-25T00:18:01.120 INFO:teuthology.orchestra.run.smithi031.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-25T00:18:01.654 INFO:teuthology.orchestra.run.smithi031.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.0 MB/s | 3.4 MB 00:01 2024-04-25T00:18:01.726 INFO:teuthology.orchestra.run.smithi040.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 823 kB/s | 1.6 MB 00:01 2024-04-25T00:18:01.854 INFO:teuthology.orchestra.run.smithi031.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T00:18:01.868 INFO:teuthology.orchestra.run.smithi040.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 1.8 MB/s | 3.4 MB 00:01 2024-04-25T00:18:01.874 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:18:01.875 INFO:teuthology.orchestra.run.smithi040.stdout:Total 8.5 MB/s | 220 MB 00:25 2024-04-25T00:18:01.896 INFO:teuthology.orchestra.run.smithi031.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 1.3 MB 00:00 2024-04-25T00:18:02.821 INFO:teuthology.orchestra.run.smithi031.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-04-25T00:18:02.824 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:18:02.824 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.0 MB/s | 220 MB 00:31 2024-04-25T00:18:03.009 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-25T00:18:03.171 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-25T00:18:03.171 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-25T00:18:03.953 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-25T00:18:04.111 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-25T00:18:04.112 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-25T00:18:04.803 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-25T00:18:04.941 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:18:05.052 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-25T00:18:05.052 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-25T00:18:05.080 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:18:05.133 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:18:05.175 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:18:05.209 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:18:05.252 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:18:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:18:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:05.806 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:05.935 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:05.973 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-25T00:18:05.973 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:18:05.973 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-25T00:18:06.127 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:18:06.178 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:18:06.206 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:06.309 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:06.332 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:18:06.395 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:18:06.416 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:18:06.435 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:18:06.607 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:18:06.649 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:18:06.756 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:18:06.845 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:18:06.890 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:18:06.990 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:18:07.024 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:18:07.067 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:18:07.100 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:18:07.128 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:18:07.197 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:18:07.250 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:18:07.280 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:18:07.316 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:18:07.393 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:18:07.427 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:18:07.462 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:18:07.490 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:18:07.562 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:18:07.605 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:18:07.680 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:18:07.818 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:18:07.866 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:18:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:18:08.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:18:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:18:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:18:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:18:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:18:08.813 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:18:08.874 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:18:08.929 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:18:09.002 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:18:09.032 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:18:09.078 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:18:09.125 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:18:09.186 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:18:09.229 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:18:09.271 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:18:09.409 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:18:09.460 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:18:09.489 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:18:09.531 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:18:09.627 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:18:09.681 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:18:09.713 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:18:09.739 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:18:09.763 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:18:09.839 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:18:09.966 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:18:10.003 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:18:10.031 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:18:10.066 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:18:10.105 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:18:10.137 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:18:10.233 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:18:10.285 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:18:10.318 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:18:10.360 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:18:10.394 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:18:10.483 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:18:10.533 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:18:10.564 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:18:10.611 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:18:10.728 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:18:10.782 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:18:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-25T00:18:11.526 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:18:11.531 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:18:11.578 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:18:11.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:18:11.621 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:18:11.632 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:18:11.682 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:18:11.707 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:18:11.721 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:18:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:18:11.779 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:18:11.906 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:18:12.085 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:18:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:18:12.146 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:18:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-25T00:18:12.208 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:12.223 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:18:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:18:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:18:12.337 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:18:12.368 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:12.392 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:18:12.425 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:18:12.461 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:12.489 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:18:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:18:12.652 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:18:12.703 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:18:12.730 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:12.836 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:18:12.884 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:18:12.919 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:18:12.932 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:18:12.941 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:18:12.961 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:18:13.012 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:18:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:18:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:18:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:18:13.400 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:18:13.440 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:18:13.475 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:18:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:18:13.552 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:18:13.651 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:18:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:18:13.683 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:18:13.711 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:18:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:18:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:18:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:18:13.784 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:18:13.844 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:18:13.872 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:18:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:18:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:18:13.986 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:18:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:18:14.018 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:18:14.062 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:18:14.090 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:18:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:18:14.166 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:18:14.214 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:18:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:18:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:18:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:18:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:18:14.293 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:18:14.358 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:18:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:18:14.403 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:18:14.436 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:18:14.441 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:18:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:18:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:18:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:18:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:18:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:18:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:18:14.749 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:18:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:18:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:18:14.807 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:18:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:18:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:18:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:18:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:18:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:18:15.072 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:18:15.098 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:18:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:18:15.157 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:18:15.190 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:18:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:18:15.233 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:18:15.272 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:18:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:18:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:18:15.438 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:18:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:18:15.476 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:18:15.477 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:18:15.533 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:18:15.547 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:18:15.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:18:15.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:18:15.664 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:18:15.700 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:18:15.737 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:18:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:18:15.819 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:18:15.863 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:18:15.905 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:18:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:18:16.050 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:18:16.093 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:18:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:18:16.132 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:18:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:18:16.174 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:18:16.199 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:18:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:18:16.273 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:18:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:18:16.315 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:18:16.355 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:18:16.381 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:18:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:18:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:18:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:18:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:18:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:18:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:18:16.623 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:18:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:18:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:18:16.691 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:18:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:18:16.725 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:18:16.741 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:18:18.166 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:18:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:18:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:18:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:18:18.246 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:18:18.280 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:18:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:18:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:18:18.405 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:18:18.448 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:18:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:18:18.474 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:18.521 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:18:18.540 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:18:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:18:18.613 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:18:18.613 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:18:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:18.648 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:18:18.749 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:18:18.933 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:18:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:18:19.035 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:18:19.254 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:18:19.309 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:18:19.343 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:18:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:18:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:18:19.729 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:18:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:18:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:18:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:18:19.930 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:18:20.623 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:18:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:18:21.178 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:18:21.250 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:18:21.278 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:18:21.312 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:18:21.640 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:18:21.682 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:18:21.775 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:18:21.805 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:18:21.866 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:18:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:18:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:18:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:18:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:18:22.364 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:18:22.394 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:18:22.449 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:18:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:18:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:18:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:18:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:18:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:18:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:18:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:18:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:18:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:18:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:18:24.840 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:24.875 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:25.448 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:25.487 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:18:25.579 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:18:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:18:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:18:25.580 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:25.613 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:18:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:18:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:18:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:18:26.429 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:18:26.429 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:18:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-25T00:18:30.295 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-25T00:18:30.296 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-25T00:18:30.296 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:30.459 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:18:30.475 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:18:30.475 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:30.475 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:18:30.476 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:31.962 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:18:31.988 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:18:31.988 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:18:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:18:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:18:31.989 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:32.213 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:18:32.236 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:18:32.236 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:32.236 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:18:32.236 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:18:32.236 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:18:32.237 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:32.689 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:18:32.712 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:18:32.712 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:32.712 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:18:32.712 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:18:32.712 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:18:32.712 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:32.827 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:18:32.884 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:18:33.128 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:18:33.181 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:18:33.261 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:18:33.349 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:18:33.489 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:18:33.531 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:18:33.563 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:18:33.591 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:18:34.041 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:18:34.073 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:18:34.105 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:18:34.167 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:18:34.204 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:18:35.244 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:18:35.281 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:18:36.250 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:18:36.281 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:18:36.384 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:18:36.556 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:18:36.583 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:18:36.607 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:18:36.607 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:36.608 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:18:36.608 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:18:36.608 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:18:36.608 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:36.659 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:18:36.671 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:18:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:18:36.824 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-04-25T00:18:37.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:37.730 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:18:37.761 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:18:37.777 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:18:37.784 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:18:37.785 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:37.785 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:18:37.785 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:18:37.785 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:18:37.785 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:37.794 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:18:37.794 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:37.794 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:18:37.794 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:37.829 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:18:37.852 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:18:37.852 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:37.852 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:18:37.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:37.896 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:18:37.919 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:18:37.919 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:37.919 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:18:37.919 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:18:37.919 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:18:37.919 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:38.230 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:18:38.254 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:18:38.254 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:38.254 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:18:38.254 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:18:38.254 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:18:38.254 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:18:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-04-25T00:18:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-04-25T00:18:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-04-25T00:18:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-04-25T00:18:38.561 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-04-25T00:18:38.561 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-04-25T00:18:38.561 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-04-25T00:18:38.561 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:18:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:18:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:18:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:39.322 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:39.597 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:18:39.621 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:18:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:18:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:18:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:18:39.622 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:40.098 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:18:40.122 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:18:40.122 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:40.122 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:18:40.122 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:18:40.122 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:18:40.123 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:40.229 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:18:40.285 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:18:40.287 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:18:40.312 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:18:40.312 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:40.312 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:18:40.312 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:18:40.312 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:18:40.312 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:40.351 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:18:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:18:40.549 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:18:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:18:40.568 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:40.568 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:18:40.568 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:18:40.568 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:18:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:18:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:18:40.780 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:18:40.925 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:18:40.966 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:18:40.997 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:18:41.033 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:18:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:18:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:18:41.071 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:41.071 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:18:41.071 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:18:41.071 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:18:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:18:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:18:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:18:41.419 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:18:41.449 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:18:41.481 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:18:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:18:41.546 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:18:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:18:41.580 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:18:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:18:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:18:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:18:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:18:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:18:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:18:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:18:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:18:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:18:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:18:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:18:42.673 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:18:42.706 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:18:42.767 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:18:42.808 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:18:42.850 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:18:42.882 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:18:42.909 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:18:43.112 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:18:43.155 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:18:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:18:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:18:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:18:43.728 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:18:43.757 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:18:43.793 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:43.841 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:43.861 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:43.865 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:18:44.036 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:18:44.067 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:18:44.092 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:18:44.092 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:44.092 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:18:44.092 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:18:44.092 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:18:44.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:44.143 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:18:44.154 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:18:44.257 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:18:44.310 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:18:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:18:44.700 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:18:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:18:44.788 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:18:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:18:44.830 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:18:44.912 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:18:44.950 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:18:44.950 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:18:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:18:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:18:45.051 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:18:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:18:45.054 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:45.054 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:18:45.054 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:18:45.054 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:18:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:45.083 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:18:45.084 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:18:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:18:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:18:45.173 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:18:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:18:45.239 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:45.256 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:18:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:18:45.288 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:18:45.312 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:18:45.312 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:45.313 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:18:45.313 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:18:45.313 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:18:45.313 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:18:45.378 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:18:45.378 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:45.378 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:18:45.379 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:45.430 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:18:45.454 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:18:45.454 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:45.454 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:18:45.454 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:18:45.455 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:18:45.455 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:45.781 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:18:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:18:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:18:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:18:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:18:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:18:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:18:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:46.307 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:18:46.330 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:18:46.331 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:46.331 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:18:46.331 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:46.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:18:46.398 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:18:46.398 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:46.398 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:18:46.398 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:18:46.399 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:18:46.399 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:46.726 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:18:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:18:46.749 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:18:46.750 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:18:46.750 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:18:46.750 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:18:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:18:50.569 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:18:50.610 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:18:50.651 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:18:50.683 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:18:50.710 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:18:50.912 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:18:50.965 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:18:51.003 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:18:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:18:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:18:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:18:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:18:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:18:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:18:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:18:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:18:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:18:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:18:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:18:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:18:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:18:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:18:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:18:51.504 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:18:51.546 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:18:51.588 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:18:51.618 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:18:51.628 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:18:51.667 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:51.688 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:18:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:18:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:18:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:18:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:52.622 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:18:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:18:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:18:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:18:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:18:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:18:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:18:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:18:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:18:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:18:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:53.194 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.195 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:18:53.197 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:18:53.198 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:18:53.199 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.200 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:18:53.201 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:18:53.202 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:18:53.203 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:18:53.204 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:18:53.205 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:18:53.206 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:18:53.206 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:18:53.206 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:18:53.206 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:18:53.206 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T00:18:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:18:53.615 DEBUG:teuthology.parallel:result is None 2024-04-25T00:18:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:18:53.673 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:18:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:18:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:18:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:18:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:18:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:18:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:18:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:18:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:58.806 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:18:58.807 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:18:58.808 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:18:58.808 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:18:58.820 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:18:58.821 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:18:58.822 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:18:58.823 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:18:58.824 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:18:58.825 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:18:58.826 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:18:58.827 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:18:58.828 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:18:58.829 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:18:58.829 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:18:58.829 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:18:58.829 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:18:58.829 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:18:59.601 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:18:59.601 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:18:59.602 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:18:59.604 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:18:59.605 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:18:59.606 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:18:59.607 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:18:59.608 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:18:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:18:59.610 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:18:59.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:18:59.612 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:18:59.613 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:18:59.613 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:19:01.058 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.060 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.061 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:19:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:19:01.063 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:19:01.064 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:19:01.065 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:19:01.066 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:19:01.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.068 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T00:19:01.069 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T00:19:01.479 DEBUG:teuthology.parallel:result is None 2024-04-25T00:19:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:19:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:19:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:19:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:19:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:19:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:19:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:19:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:19:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:19:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:19:01.703 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T00:19:02.074 DEBUG:teuthology.parallel:result is None 2024-04-25T00:19:02.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:19:02.218 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:19:02.244 INFO:teuthology.orchestra.run.smithi031.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:19:02.244 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:19:02.244 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:19:02.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:19:02.474 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:19:02.496 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:19:02.496 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:19:02.496 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:19:02.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:19:02.716 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:19:02.742 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:19:02.742 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:19:02.742 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:19:02.744 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:19:02.744 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:19:02.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:19:02.772 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:19:02.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:19:02.799 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:19:02.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:19:02.830 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:19:02.830 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:19:02.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:19:02.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:19:02.921 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:19:02.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:19:02.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:19:03.010 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:19:03.011 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:19:03.039 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:19:03.108 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:19:03.108 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:19:03.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:19:03.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:19:03.198 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:19:03.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:19:03.222 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:19:03.286 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:19:03.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:19:03.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:19:03.381 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:19:03.382 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:19:03.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:19:03.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:19:03.471 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:19:03.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:19:03.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:19:03.560 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:19:03.561 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:19:03.588 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:19:03.655 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T00:19:03.755 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T00:19:03.755 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:19:03.756 INFO:tasks.cephadm:Cluster fsid is 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:19:03.756 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T00:19:03.756 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.40', 'mon.c': '172.21.15.169'} 2024-04-25T00:19:03.756 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-04-25T00:19:03.756 INFO:tasks.cephadm:First mgr is x 2024-04-25T00:19:03.757 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T00:19:03.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-04-25T00:19:03.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname $(hostname -s) 2024-04-25T00:19:03.808 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-04-25T00:19:03.835 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T00:19:03.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:19:04.055 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T00:19:04.273 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:19:04.275 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-25T00:19:04.275 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:05.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:19 /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:05.279 DEBUG:teuthology.orchestra.run.smithi040:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:06.318 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:19 /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:06.319 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:07.463 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:19 /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:07.463 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:07.483 DEBUG:teuthology.orchestra.run.smithi040:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:07.502 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:19:07.535 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-25T00:19:07.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:19:07.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:19:07.547 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:19:09.175 INFO:teuthology.orchestra.run.smithi040.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:19:09.216 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:19:09.259 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: "repo_digests": [ 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-25T00:19:36.921 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T00:20:00.856 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:20:00.857 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:20:00.857 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:20:00.857 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-04-25T00:20:00.857 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:20:00.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:20:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:20:03.590 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T00:20:03.590 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:20:03.590 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:20:03.590 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-04-25T00:20:03.591 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:20:03.591 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-04-25T00:20:03.591 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T00:20:03.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-04-25T00:20:03.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph 2024-04-25T00:20:03.682 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-04-25T00:20:03.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-04-25T00:20:03.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /etc/ceph 2024-04-25T00:20:03.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-04-25T00:20:03.792 INFO:tasks.cephadm:Writing seed config... 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T00:20:03.793 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T00:20:03.794 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T00:20:03.795 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T00:20:03.796 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T00:20:03.797 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:20:03.797 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T00:20:03.814 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6c091b24-0299-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T00:20:03.815 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T00:20:03.857 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T00:20:03.899 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T00:20:03.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:20:04.103 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:20:04.103 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '6c091b24-0299-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-04-25T00:20:04.104 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T00:20:04.104 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-04-25T00:20:04.130 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:20:04.130 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-04-25T00:20:04.130 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-04-25T00:20:04.137 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:20:04.137 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:20:04.144 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:20:04.144 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T00:20:04.150 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-25T00:20:04.156 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-25T00:20:04.157 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:20:04.157 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-04-25T00:20:04.181 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:20:04.181 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T00:20:04.181 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-04-25T00:20:04.182 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-04-25T00:20:04.188 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:20:04.188 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:20:04.194 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:20:04.194 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T00:20:04.200 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-25T00:20:04.207 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-25T00:20:04.207 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:20:04.207 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-04-25T00:20:04.207 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:04.207 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140618988229536 on /run/cephadm/6c091b24-0299-11ef-bc93-c7b262605968.lock 2024-04-25T00:20:04.207 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140618988229536 acquired on /run/cephadm/6c091b24-0299-11ef-bc93-c7b262605968.lock 2024-04-25T00:20:04.208 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-04-25T00:20:04.208 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-04-25T00:20:04.208 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-25T00:20:04.211 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-04-25T00:20:04.211 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-04-25T00:20:04.214 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T00:20:04.214 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T00:20:04.214 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T00:20:04.216 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T00:20:04.216 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T00:20:04.216 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:20:04.216 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T00:20:04.216 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-25T00:20:04.216 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:20:04.217 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-25T00:20:04.217 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-25T00:20:04.217 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T00:20:04.218 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T00:20:04.218 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:20:04.941 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:20:04.941 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:20:04.941 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T00:20:04.941 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-25T00:20:04.942 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T00:20:04.942 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:20:04.942 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T00:20:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:20:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:20:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T00:20:06.808 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-04-25T00:20:06.808 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-04-25T00:20:07.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC3oSlmgUNfBxAAks9Az3udjBPlmxOOyYdWHg== 2024-04-25T00:20:08.709 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC4oSlmkjCmBhAAwUXKaBwLD4ZMsmoDrWjmqQ== 2024-04-25T00:20:09.633 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC5oSlmHa2HAxAAuwz0pufKBbkCFlEe45qZVw== 2024-04-25T00:20:09.634 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:20:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:20:10.699 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-04-25T00:20:11.653 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-04-25T00:20:11.863 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T00:20:12.011 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T00:20:12.296 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6c091b24-0299-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-6c091b24-0299-11ef-bc93-c7b262605968.target. 2024-04-25T00:20:12.296 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6c091b24-0299-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-6c091b24-0299-11ef-bc93-c7b262605968.target. 2024-04-25T00:20:12.529 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:20:12.529 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T00:20:12.691 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6c091b24-0299-11ef-bc93-c7b262605968.target.wants/ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-6c091b24-0299-11ef-bc93-c7b262605968@.service. 2024-04-25T00:20:13.195 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:20:13.195 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T00:20:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:20:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T00:20:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T00:20:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:20:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-04-25T00:20:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-04-25T00:20:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:13 smithi031 systemd[1]: Started Ceph mon.a for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: monmap epoch 1 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: last_changed 2024-04-25T00:20:09.931607+0000 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: min_mon_release 19 (squid) 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: election_strategy: 1 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: fsmap 2024-04-25T00:20:14.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:20:14.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: mgrmap e1: no daemons active 2024-04-25T00:20:14.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:14 smithi031 ceph-mon[28787]: from='client.? 172.21.15.31:0/3030728344' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T00:20:14.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.551601s) 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T00:20:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.611+0000 7f84391ee700 1 Processor -- start 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.611+0000 7f84391ee700 1 -- start start 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.611+0000 7f84391ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843410a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.611+0000 7f84391ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f843407be50 con 0x7f84341085b0 2024-04-25T00:20:14.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8432d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843410a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8432d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843410a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55770/0 (socket says 172.21.15.31:55770) 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8432d9d700 1 -- 172.21.15.31:0/2423425986 learned_addr learned my addr 172.21.15.31:0/2423425986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8432d9d700 1 -- 172.21.15.31:0/2423425986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843410aee0 con 0x7f84341085b0 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8432d9d700 1 --2- 172.21.15.31:0/2423425986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843410a9a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f841c0098a0 tx=0x7f841c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4dba5194cfcdea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8431d9b700 1 -- 172.21.15.31:0/2423425986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c004160 con 0x7f84341085b0 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8431d9b700 1 -- 172.21.15.31:0/2423425986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c004300 con 0x7f84341085b0 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.612+0000 7f8431d9b700 1 -- 172.21.15.31:0/2423425986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c0036a0 con 0x7f84341085b0 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 -- 172.21.15.31:0/2423425986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 msgr2=0x7f843410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 --2- 172.21.15.31:0/2423425986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843410a9a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f841c0098a0 tx=0x7f841c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 -- 172.21.15.31:0/2423425986 shutdown_connections 2024-04-25T00:20:14.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 --2- 172.21.15.31:0/2423425986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843410a9a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 -- 172.21.15.31:0/2423425986 >> 172.21.15.31:0/2423425986 conn(0x7f8434101fc0 msgr2=0x7f8434104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 -- 172.21.15.31:0/2423425986 shutdown_connections 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.613+0000 7f84391ee700 1 -- 172.21.15.31:0/2423425986 wait complete. 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f84391ee700 1 Processor -- start 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f84391ee700 1 -- start start 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f84391ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f84391ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f843410b3b0 con 0x7f84341085b0 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f8432d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f8432d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55778/0 (socket says 172.21.15.31:55778) 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.614+0000 7f8432d9d700 1 -- 172.21.15.31:0/3030728344 learned_addr learned my addr 172.21.15.31:0/3030728344 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f8432d9d700 1 -- 172.21.15.31:0/3030728344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843419b7c0 con 0x7f84341085b0 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f8432d9d700 1 --2- 172.21.15.31:0/3030728344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843419b280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:14.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c003db0 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c011630 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843419ba50 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c011950 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843419e740 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f841c011af0 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.615+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f841c01c790 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.617+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8434063890 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.619+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f841c032020 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.693+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f843402dcb0 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.694+0000 7f842bfff700 1 -- 172.21.15.31:0/3030728344 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f841c004260 con 0x7f84341085b0 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 msgr2=0x7f843419b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 --2- 172.21.15.31:0/3030728344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843419b280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c004750 comp rx=0 tx=0).stop 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 shutdown_connections 2024-04-25T00:20:14.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 --2- 172.21.15.31:0/3030728344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84341085b0 0x7f843419b280 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 >> 172.21.15.31:0/3030728344 conn(0x7f8434101fc0 msgr2=0x7f8434102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:14.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 shutdown_connections 2024-04-25T00:20:14.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:13.695+0000 7f84391ee700 1 -- 172.21.15.31:0/3030728344 wait complete. 2024-04-25T00:20:14.262 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-04-25T00:20:14.262 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T00:20:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:15 smithi031 ceph-mon[28787]: from='client.? 172.21.15.31:0/2576376258' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:20:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:15 smithi031 ceph-mon[28787]: from='client.? 172.21.15.31:0/2576376258' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T00:20:15.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:15.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:20:15.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:15.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-25T00:20:15.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.618+0000 7f7e7c2e8700 1 Processor -- start 2024-04-25T00:20:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7c2e8700 1 -- start start 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7c2e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7c2e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7407c850 con 0x7f7e7407de20 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7a084700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7a084700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55784/0 (socket says 172.21.15.31:55784) 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7a084700 1 -- 172.21.15.31:0/1831143217 learned_addr learned my addr 172.21.15.31:0/1831143217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.619+0000 7f7e7a084700 1 -- 172.21.15.31:0/1831143217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7407c9d0 con 0x7f7e7407de20 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7a084700 1 --2- 172.21.15.31:0/1831143217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7407c280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098a0 tx=0x7f7e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d034afd926a97f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e79082700 1 -- 172.21.15.31:0/1831143217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e68004160 con 0x7f7e7407de20 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e79082700 1 -- 172.21.15.31:0/1831143217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e68004300 con 0x7f7e7407de20 2024-04-25T00:20:15.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e79082700 1 -- 172.21.15.31:0/1831143217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e680036a0 con 0x7f7e7407de20 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/1831143217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 msgr2=0x7f7e7407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 --2- 172.21.15.31:0/1831143217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7407c280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098a0 tx=0x7f7e68009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/1831143217 shutdown_connections 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 --2- 172.21.15.31:0/1831143217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7407c280 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/1831143217 >> 172.21.15.31:0/1831143217 conn(0x7f7e74103180 msgr2=0x7f7e741055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/1831143217 shutdown_connections 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.620+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/1831143217 wait complete. 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7c2e8700 1 Processor -- start 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7c2e8700 1 -- start start 2024-04-25T00:20:15.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7c2e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7c2e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7407cf00 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7a084700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7a084700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55800/0 (socket says 172.21.15.31:55800) 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.621+0000 7f7e7a084700 1 -- 172.21.15.31:0/2576376258 learned_addr learned my addr 172.21.15.31:0/2576376258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e7a084700 1 -- 172.21.15.31:0/2576376258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7419d9a0 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e7a084700 1 --2- 172.21.15.31:0/2576376258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7419d460 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7e68000c00 tx=0x7f7e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e68003db0 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e68011630 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7419dc30 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e68011950 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741a0920 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7e68011af0 con 0x7f7e7407de20 2024-04-25T00:20:15.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.622+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7e6801c450 con 0x7f7e7407de20 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.623+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e74063890 con 0x7f7e7407de20 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.625+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7e68032020 con 0x7f7e7407de20 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.696+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7e7402dcb0 con 0x7f7e7407de20 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.699+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f7e68004260 con 0x7f7e7407de20 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.700+0000 7f7e677fe700 1 -- 172.21.15.31:0/2576376258 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7e6801b450 con 0x7f7e7407de20 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 msgr2=0x7f7e7419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 --2- 172.21.15.31:0/2576376258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7419d460 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7e68000c00 tx=0x7f7e68004750 comp rx=0 tx=0).stop 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 shutdown_connections 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 --2- 172.21.15.31:0/2576376258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e7407de20 0x7f7e7419d460 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 >> 172.21.15.31:0/2576376258 conn(0x7f7e74103180 msgr2=0x7f7e74104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 shutdown_connections 2024-04-25T00:20:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:14.701+0000 7f7e7c2e8700 1 -- 172.21.15.31:0/2576376258 wait complete. 2024-04-25T00:20:15.295 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-04-25T00:20:16.330 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:16 smithi031 ceph-mon[28787]: from='client.? 172.21.15.31:0/1003436130' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:20:16.336 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.689+0000 7f446565e700 1 Processor -- start 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.689+0000 7f446565e700 1 -- start start 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.689+0000 7f446565e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446010aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.689+0000 7f446565e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f446007bdf0 con 0x7f4460108650 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446010aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446010aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55802/0 (socket says 172.21.15.31:55802) 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445effd700 1 -- 172.21.15.31:0/3939414548 learned_addr learned my addr 172.21.15.31:0/3939414548 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445effd700 1 -- 172.21.15.31:0/3939414548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446010af80 con 0x7f4460108650 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445effd700 1 --2- 172.21.15.31:0/3939414548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446010aa40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4458009e80 tx=0x7f445800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447269b6d52c3f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445dffb700 1 -- 172.21.15.31:0/3939414548 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4458017070 con 0x7f4460108650 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445dffb700 1 -- 172.21.15.31:0/3939414548 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f445800b8c0 con 0x7f4460108650 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.690+0000 7f445dffb700 1 -- 172.21.15.31:0/3939414548 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4458004520 con 0x7f4460108650 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 -- 172.21.15.31:0/3939414548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 msgr2=0x7f446010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 --2- 172.21.15.31:0/3939414548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446010aa40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4458009e80 tx=0x7f445800b330 comp rx=0 tx=0).stop 2024-04-25T00:20:16.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 -- 172.21.15.31:0/3939414548 shutdown_connections 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 --2- 172.21.15.31:0/3939414548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446010aa40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 -- 172.21.15.31:0/3939414548 >> 172.21.15.31:0/3939414548 conn(0x7f4460102020 msgr2=0x7f4460104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 -- 172.21.15.31:0/3939414548 shutdown_connections 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.691+0000 7f446565e700 1 -- 172.21.15.31:0/3939414548 wait complete. 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f446565e700 1 Processor -- start 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f446565e700 1 -- start start 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f446565e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f446565e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f446010bef0 con 0x7f4460108650 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f445effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f445effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55804/0 (socket says 172.21.15.31:55804) 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f445effd700 1 -- 172.21.15.31:0/1003436130 learned_addr learned my addr 172.21.15.31:0/1003436130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.692+0000 7f445effd700 1 -- 172.21.15.31:0/1003436130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446019bf80 con 0x7f4460108650 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f445effd700 1 --2- 172.21.15.31:0/1003436130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446019ba40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f445800b7b0 tx=0x7f4458004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4458020020 con 0x7f4460108650 2024-04-25T00:20:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f445800bcb0 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4458017040 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446019c210 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446019ef00 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4458023020 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.693+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f445800f050 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.694+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4460063890 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.696+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f445802a020 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.766+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f446019f0d0 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.767+0000 7f4457fff700 1 -- 172.21.15.31:0/1003436130 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f445800b860 con 0x7f4460108650 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.767+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 msgr2=0x7f446019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.767+0000 7f446565e700 1 --2- 172.21.15.31:0/1003436130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446019ba40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f445800b7b0 tx=0x7f4458004670 comp rx=0 tx=0).stop 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.768+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 shutdown_connections 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.768+0000 7f446565e700 1 --2- 172.21.15.31:0/1003436130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4460108650 0x7f446019ba40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:16.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.768+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 >> 172.21.15.31:0/1003436130 conn(0x7f4460102020 msgr2=0x7f4460102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:16.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.768+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 shutdown_connections 2024-04-25T00:20:16.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:15.768+0000 7f446565e700 1 -- 172.21.15.31:0/1003436130 wait complete. 2024-04-25T00:20:16.340 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-04-25T00:20:16.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:16 smithi031 systemd[1]: Stopping Ceph mon.a for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:20:16.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:16 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a[28782]: 2024-04-25T00:20:16.599+0000 7efd32a42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:16.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:16 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a[28782]: 2024-04-25T00:20:16.599+0000 7efd32a42700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T00:20:17.108 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:16 smithi031 podman[29042]: 2024-04-25 00:20:16.856432547 +0000 UTC m=+0.356301091 container died 20f6d9c1f2d61cf477f3201bca1317bbe48a18109bf5d166b33bd698fdf85a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T00:20:17.366 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 podman[29042]: 2024-04-25 00:20:17.199381771 +0000 UTC m=+0.699250285 container cleanup 20f6d9c1f2d61cf477f3201bca1317bbe48a18109bf5d166b33bd698fdf85a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:20:17.367 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 bash[29042]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:20:17.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 podman[29056]: 2024-04-25 00:20:17.36682296 +0000 UTC m=+0.504673375 container remove 20f6d9c1f2d61cf477f3201bca1317bbe48a18109bf5d166b33bd698fdf85a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T00:20:17.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 systemd[1]: ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T00:20:17.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 systemd[1]: Stopped Ceph mon.a for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:20:17.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 systemd[1]: Starting Ceph mon.a for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:20:18.041 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 podman[29148]: 2024-04-25 00:20:17.792570434 +0000 UTC m=+0.111005839 container create 5218831c95b631d2c27e7858d4ccba3740a89f9da9cb58c5ceec0ef4edc18f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 podman[29148]: 2024-04-25 00:20:17.699438589 +0000 UTC m=+0.017873992 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 podman[29148]: 2024-04-25 00:20:17.969984064 +0000 UTC m=+0.288419467 container init 5218831c95b631d2c27e7858d4ccba3740a89f9da9cb58c5ceec0ef4edc18f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 podman[29148]: 2024-04-25 00:20:17.97282344 +0000 UTC m=+0.291258843 container start 5218831c95b631d2c27e7858d4ccba3740a89f9da9cb58c5ceec0ef4edc18f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 ceph-mon[29167]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 ceph-mon[29167]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:20:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:17 smithi031 ceph-mon[29167]: pidfile_write: ignore empty --pid-file 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: load: jerasure load: lrc 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Git sha 0 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: DB SUMMARY 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: DB Session ID: 3F47VCGQD0WHQS0IYQIJ 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104910 ; 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:20:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.env: 0x560e07f2cc60 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.info_log: 0x560e09defaa0 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.statistics: (nil) 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.use_fsync: 0 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:20:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.db_log_dir: 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.wal_dir: 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:20:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.write_buffer_manager: 0x560e09dc21e0 2024-04-25T00:20:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:20:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:20:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:20:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:20:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:20:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:20:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:20:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:20:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.unordered_write: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.row_cache: None 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.wal_filter: None 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.wal_compression: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:20:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:20:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_open_files: -1 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:20:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Compression algorithms supported: 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kZSTD supported: 0 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:20:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.merge_operator: 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_filter: None 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e09defba0) 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:20:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x560e09df3090 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-04-25T00:20:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-04-25T00:20:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression: NoCompression 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.num_levels: 7 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:20:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:20:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:20:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:20:18.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:20:18.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.ttl: 2592000 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:20:18.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1be3d26f-f149-4ed2-a460-6e5697a2fc33 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714004418002122, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714004418003115, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97943, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14193, "raw_average_key_size": 49, "raw_value_size": 90222, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714004418, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1be3d26f-f149-4ed2-a460-6e5697a2fc33", "db_session_id": "3F47VCGQD0WHQS0IYQIJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714004418003200, "job": 1, "event": "recovery_finished"} 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e09ee4000 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: DB pointer 0x560e09ece000 2024-04-25T00:20:18.083 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 99.52 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 99.52 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:20:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:20:18.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 31.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 31.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x560e09df3090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???) e1 preinit fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).mds e1 new map 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).mds e1 print_map 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-04-25T00:20:18.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 bash[29148]: 5218831c95b631d2c27e7858d4ccba3740a89f9da9cb58c5ceec0ef4edc18f32 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 systemd[1]: Started Ceph mon.a for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: monmap epoch 1 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:18.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: last_changed 2024-04-25T00:20:09.931607+0000 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: min_mon_release 19 (squid) 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: election_strategy: 1 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: fsmap 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:18 smithi031 ceph-mon[29167]: mgrmap e1: no daemons active 2024-04-25T00:20:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.461+0000 7f3f4b0c1700 1 Processor -- start 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.461+0000 7f3f4b0c1700 1 -- start start 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.461+0000 7f3f4b0c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f4407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.461+0000 7f3f4b0c1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4407c8b0 con 0x7f3f4407de80 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f48e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f4407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f48e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f4407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55814/0 (socket says 172.21.15.31:55814) 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f48e5d700 1 -- 172.21.15.31:0/839421391 learned_addr learned my addr 172.21.15.31:0/839421391 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f48e5d700 1 -- 172.21.15.31:0/839421391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4407ca30 con 0x7f3f4407de80 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f48e5d700 1 --2- 172.21.15.31:0/839421391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f4407c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009e80 tx=0x7f3f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2196415f990f43c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f437fe700 1 -- 172.21.15.31:0/839421391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c017070 con 0x7f3f4407de80 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.462+0000 7f3f437fe700 1 -- 172.21.15.31:0/839421391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3f3c00b8c0 con 0x7f3f4407de80 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f437fe700 1 -- 172.21.15.31:0/839421391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c004520 con 0x7f3f4407de80 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/839421391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 msgr2=0x7f3f4407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 --2- 172.21.15.31:0/839421391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f4407c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009e80 tx=0x7f3f3c00b330 comp rx=0 tx=0).stop 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/839421391 shutdown_connections 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 --2- 172.21.15.31:0/839421391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f4407c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/839421391 >> 172.21.15.31:0/839421391 conn(0x7f3f44103180 msgr2=0x7f3f441055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/839421391 shutdown_connections 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.463+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/839421391 wait complete. 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f4b0c1700 1 Processor -- start 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f4b0c1700 1 -- start start 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f4b0c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f441a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f4b0c1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44109d50 con 0x7f3f4407de80 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f48e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f441a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f48e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f441a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55826/0 (socket says 172.21.15.31:55826) 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f48e5d700 1 -- 172.21.15.31:0/1530823396 learned_addr learned my addr 172.21.15.31:0/1530823396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f48e5d700 1 -- 172.21.15.31:0/1530823396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f441a25f0 con 0x7f3f4407de80 2024-04-25T00:20:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.464+0000 7f3f48e5d700 1 --2- 172.21.15.31:0/1530823396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f441a20b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c003f60 tx=0x7f3f3c004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c020020 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3f3c00bcb0 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f441a2880 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c017040 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f441a5570 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f3c023020 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.465+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f3c00f050 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.466+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4419b640 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.468+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3f3c02a020 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.538+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3f4402dd60 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.541+0000 7f3f41ffb700 1 -- 172.21.15.31:0/1530823396 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3f3c00b860 con 0x7f3f4407de80 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.541+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 msgr2=0x7f3f441a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.541+0000 7f3f4b0c1700 1 --2- 172.21.15.31:0/1530823396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f441a20b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c003f60 tx=0x7f3f3c004670 comp rx=0 tx=0).stop 2024-04-25T00:20:19.077 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.541+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 shutdown_connections 2024-04-25T00:20:19.078 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.541+0000 7f3f4b0c1700 1 --2- 172.21.15.31:0/1530823396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4407de80 0x7f3f441a20b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.078 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.541+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 >> 172.21.15.31:0/1530823396 conn(0x7f3f44103180 msgr2=0x7f3f44105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.078 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.542+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 shutdown_connections 2024-04-25T00:20:19.078 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:18.542+0000 7f3f4b0c1700 1 -- 172.21.15.31:0/1530823396 wait complete. 2024-04-25T00:20:19.078 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T00:20:19.100 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:20:19.100 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-04-25T00:20:19.100 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T00:20:19.101 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T00:20:19.369 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:20:19.369 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T00:20:19.534 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6c091b24-0299-11ef-bc93-c7b262605968.target.wants/ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-6c091b24-0299-11ef-bc93-c7b262605968@.service. 2024-04-25T00:20:19.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:19 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1530823396' entity='client.admin' 2024-04-25T00:20:19.761 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:19 smithi031 systemd[1]: Starting Ceph mgr.x for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:20:19.761 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:19 smithi031 podman[29383]: 2024-04-25 00:20:19.760856251 +0000 UTC m=+0.098319739 container create 9b1f6788656b831ae01c20f6b5de954f634068a950e31a5d7349dc7bce1605e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:20:20.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:19 smithi031 podman[29383]: 2024-04-25 00:20:19.68080833 +0000 UTC m=+0.018271769 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:20:20.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:19 smithi031 podman[29383]: 2024-04-25 00:20:19.996346384 +0000 UTC m=+0.333809804 container init 9b1f6788656b831ae01c20f6b5de954f634068a950e31a5d7349dc7bce1605e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:20.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:19 smithi031 podman[29383]: 2024-04-25 00:20:19.998977866 +0000 UTC m=+0.336441285 container start 9b1f6788656b831ae01c20f6b5de954f634068a950e31a5d7349dc7bce1605e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:20:20.094 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:20:20.094 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T00:20:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:20:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T00:20:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T00:20:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:20:20.109 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:20:20.109 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T00:20:20.117 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:20:20.117 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T00:20:20.117 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T00:20:20.117 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T00:20:20.117 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-04-25T00:20:20.117 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-04-25T00:20:20.348 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 bash[29383]: 9b1f6788656b831ae01c20f6b5de954f634068a950e31a5d7349dc7bce1605e3 2024-04-25T00:20:20.349 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 systemd[1]: Started Ceph mgr.x for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:20:20.349 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:20.155+0000 7f32675dc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:20:20.349 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:20.347+0000 7f32675dc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:20:20.625 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:20.452+0000 7f32675dc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:20:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:20 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1291302604' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:20:21.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:20.785+0000 7f32675dc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:20:21.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:20 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:20.873+0000 7f32675dc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:20:21.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:21.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:20:21.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:20:21.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:20:21.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:20:21.337 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:20:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:20:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:20:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:20:13.144738+0000", 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6d25ce700 1 Processor -- start 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6d25ce700 1 -- start start 2024-04-25T00:20:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6d25ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6d25ce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc07bdf0 con 0x7fe6cc106400 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55848/0 (socket says 172.21.15.31:55848) 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.653+0000 7fe6cbfff700 1 -- 172.21.15.31:0/3107667724 learned_addr learned my addr 172.21.15.31:0/3107667724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.654+0000 7fe6cbfff700 1 -- 172.21.15.31:0/3107667724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc106d60 con 0x7fe6cc106400 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.654+0000 7fe6cbfff700 1 --2- 172.21.15.31:0/3107667724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc106820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00098a0 tx=0x7fe6c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289961ff397c7da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.654+0000 7fe6caffd700 1 -- 172.21.15.31:0/3107667724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c0004160 con 0x7fe6cc106400 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.654+0000 7fe6caffd700 1 -- 172.21.15.31:0/3107667724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe6c0004300 con 0x7fe6cc106400 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.654+0000 7fe6caffd700 1 -- 172.21.15.31:0/3107667724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c0011a80 con 0x7fe6cc106400 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 -- 172.21.15.31:0/3107667724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 msgr2=0x7fe6cc106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 --2- 172.21.15.31:0/3107667724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc106820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00098a0 tx=0x7fe6c0009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 -- 172.21.15.31:0/3107667724 shutdown_connections 2024-04-25T00:20:21.342 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 --2- 172.21.15.31:0/3107667724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc106820 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.345 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 -- 172.21.15.31:0/3107667724 >> 172.21.15.31:0/3107667724 conn(0x7fe6cc101fc0 msgr2=0x7fe6cc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 -- 172.21.15.31:0/3107667724 shutdown_connections 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.655+0000 7fe6d25ce700 1 -- 172.21.15.31:0/3107667724 wait complete. 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6d25ce700 1 Processor -- start 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6d25ce700 1 -- start start 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6d25ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6d25ce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc107d90 con 0x7fe6cc106400 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc19dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55850/0 (socket says 172.21.15.31:55850) 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6cbfff700 1 -- 172.21.15.31:0/1291302604 learned_addr learned my addr 172.21.15.31:0/1291302604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6cbfff700 1 -- 172.21.15.31:0/1291302604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc19e190 con 0x7fe6cc106400 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.656+0000 7fe6cbfff700 1 --2- 172.21.15.31:0/1291302604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc19dc50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0006af0 tx=0x7fe6c0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c00039e0 con 0x7fe6cc106400 2024-04-25T00:20:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe6c0003b80 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc19e420 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c00196a0 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a1110 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe6c0019d00 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.657+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe6c0024730 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.658+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc063890 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.660+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe6c0043dd0 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.734+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe6cc02dd80 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.734+0000 7fe6c97fa700 1 -- 172.21.15.31:0/1291302604 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe6c0032020 con 0x7fe6cc106400 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 msgr2=0x7fe6cc19dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 --2- 172.21.15.31:0/1291302604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc19dc50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0006af0 tx=0x7fe6c0011c40 comp rx=0 tx=0).stop 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 shutdown_connections 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 --2- 172.21.15.31:0/1291302604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6cc106400 0x7fe6cc19dc50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 >> 172.21.15.31:0/1291302604 conn(0x7fe6cc101fc0 msgr2=0x7fe6cc194910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 shutdown_connections 2024-04-25T00:20:21.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:20.735+0000 7fe6d25ce700 1 -- 172.21.15.31:0/1291302604 wait complete. 2024-04-25T00:20:21.348 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-04-25T00:20:22.489 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:22.140+0000 7f32675dc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:20:22.490 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:22.314+0000 7f32675dc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:20:22.490 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:22.489+0000 7f32675dc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:20:23.089 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:22.784+0000 7f32675dc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:20:23.089 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:22.879+0000 7f32675dc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:20:23.089 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:23 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:23.088+0000 7f32675dc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:20:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:23 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:23.285+0000 7f32675dc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:20:24.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:23 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2890057071' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:20:24.023 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:23 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:23.926+0000 7f32675dc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:20:24.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:24.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:20:24.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:20:24.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:20:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:20:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:20:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:20:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:20:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:20:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:20:24.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:20:24.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:20:13.144738+0000", 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:24.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f29474700 1 Processor -- start 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f29474700 1 -- start start 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f29474700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f24107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f29474700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f24108520 con 0x7f6f24107b30 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f24107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f24107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39564/0 (socket says 172.21.15.31:39564) 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.770+0000 7f6f22ffd700 1 -- 172.21.15.31:0/1611602387 learned_addr learned my addr 172.21.15.31:0/1611602387 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.771+0000 7f6f22ffd700 1 -- 172.21.15.31:0/1611602387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f24108d30 con 0x7f6f24107b30 2024-04-25T00:20:24.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.771+0000 7f6f22ffd700 1 --2- 172.21.15.31:0/1611602387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f24107f50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f140098a0 tx=0x7f6f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34b4f323d4e2f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.771+0000 7f6f21ffb700 1 -- 172.21.15.31:0/1611602387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f14004160 con 0x7f6f24107b30 2024-04-25T00:20:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.771+0000 7f6f21ffb700 1 -- 172.21.15.31:0/1611602387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6f14004300 con 0x7f6f24107b30 2024-04-25T00:20:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.771+0000 7f6f21ffb700 1 -- 172.21.15.31:0/1611602387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f14011a80 con 0x7f6f24107b30 2024-04-25T00:20:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 -- 172.21.15.31:0/1611602387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 msgr2=0x7f6f24107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 --2- 172.21.15.31:0/1611602387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f24107f50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f140098a0 tx=0x7f6f14009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 -- 172.21.15.31:0/1611602387 shutdown_connections 2024-04-25T00:20:24.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 --2- 172.21.15.31:0/1611602387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f24107f50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:24.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 -- 172.21.15.31:0/1611602387 >> 172.21.15.31:0/1611602387 conn(0x7f6f24103110 msgr2=0x7f6f24105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:24.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 -- 172.21.15.31:0/1611602387 shutdown_connections 2024-04-25T00:20:24.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.772+0000 7f6f29474700 1 -- 172.21.15.31:0/1611602387 wait complete. 2024-04-25T00:20:24.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f29474700 1 Processor -- start 2024-04-25T00:20:24.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f29474700 1 -- start start 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f29474700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f2407db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f29474700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410c0e0 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f2407db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f2407db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39574/0 (socket says 172.21.15.31:39574) 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.773+0000 7f6f22ffd700 1 -- 172.21.15.31:0/2890057071 learned_addr learned my addr 172.21.15.31:0/2890057071 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f22ffd700 1 -- 172.21.15.31:0/2890057071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2407e050 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f22ffd700 1 --2- 172.21.15.31:0/2890057071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f2407db10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000c00 tx=0x7f6f14011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f140039f0 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2407c240 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6f14003b90 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f1401a740 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f2407c780 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.774+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6f1402d3f0 con 0x7f6f24107b30 2024-04-25T00:20:24.520 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.775+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f14019700 con 0x7f6f24107b30 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.775+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f24063890 con 0x7f6f24107b30 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.777+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6f14024af0 con 0x7f6f24107b30 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.852+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6f2407cac0 con 0x7f6f24107b30 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.852+0000 7f6f0bfff700 1 -- 172.21.15.31:0/2890057071 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6f14024440 con 0x7f6f24107b30 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 msgr2=0x7f6f2407db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 --2- 172.21.15.31:0/2890057071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f2407db10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000c00 tx=0x7f6f14011b00 comp rx=0 tx=0).stop 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 shutdown_connections 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 --2- 172.21.15.31:0/2890057071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f24107b30 0x7f6f2407db10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 >> 172.21.15.31:0/2890057071 conn(0x7f6f24103110 msgr2=0x7f6f24103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:24.521 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 shutdown_connections 2024-04-25T00:20:24.522 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:23.853+0000 7f6f29474700 1 -- 172.21.15.31:0/2890057071 wait complete. 2024-04-25T00:20:24.522 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-04-25T00:20:24.675 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:24 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:24.368+0000 7f32675dc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:20:24.675 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:24 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:24.467+0000 7f32675dc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:20:24.675 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:24 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:24.674+0000 7f32675dc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:20:25.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:25.232+0000 7f32675dc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:20:25.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:25.325+0000 7f32675dc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:20:25.934 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:25.575+0000 7f32675dc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:20:25.934 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:25.715+0000 7f32675dc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:20:25.934 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:25.813+0000 7f32675dc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:20:25.934 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:25.933+0000 7f32675dc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:20:26.272 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:26 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:26.027+0000 7f32675dc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:20:27.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:26 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:26.873+0000 7f32675dc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:20:27.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:27.095+0000 7f32675dc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:20:27.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: Activating manager daemon x 2024-04-25T00:20:27.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: mgrmap e2: x(active, starting, since 0.00326267s) 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: Manager daemon x is now available 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' 2024-04-25T00:20:27.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:27 smithi031 ceph-mon[29167]: from='mgr.14100 172.21.15.31:0/60642701' entity='mgr.x' 2024-04-25T00:20:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:29 smithi031 ceph-mon[29167]: mgrmap e3: x(active, since 1.00807s) 2024-04-25T00:20:30.300 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:30 smithi031 ceph-mon[29167]: mgrmap e4: x(active, since 2s) 2024-04-25T00:20:30.300 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:30 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3299504787' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:20:36.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:36.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:20:36.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:20:36.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:20:36.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:20:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:20:36.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:20:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:20:36.229 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:20:13.144738+0000", 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5ac046d700 1 Processor -- start 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5ac046d700 1 -- start start 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5ac046d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab810aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5ac046d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab807be10 con 0x7f5ab8108650 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5abe209700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab810aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5abe209700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab810aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39632/0 (socket says 172.21.15.31:39632) 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5abe209700 1 -- 172.21.15.31:0/4118614506 learned_addr learned my addr 172.21.15.31:0/4118614506 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.798+0000 7f5abe209700 1 -- 172.21.15.31:0/4118614506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab810af80 con 0x7f5ab8108650 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5abe209700 1 --2- 172.21.15.31:0/4118614506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab810aa40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098a0 tx=0x7f5ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a53e2a129730c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5abd207700 1 -- 172.21.15.31:0/4118614506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ab4004160 con 0x7f5ab8108650 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5abd207700 1 -- 172.21.15.31:0/4118614506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5ab4004300 con 0x7f5ab8108650 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5abd207700 1 -- 172.21.15.31:0/4118614506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ab4011a80 con 0x7f5ab8108650 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 -- 172.21.15.31:0/4118614506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 msgr2=0x7f5ab810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 --2- 172.21.15.31:0/4118614506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab810aa40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098a0 tx=0x7f5ab4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 -- 172.21.15.31:0/4118614506 shutdown_connections 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 --2- 172.21.15.31:0/4118614506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab810aa40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 -- 172.21.15.31:0/4118614506 >> 172.21.15.31:0/4118614506 conn(0x7f5ab8102060 msgr2=0x7f5ab8104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 -- 172.21.15.31:0/4118614506 shutdown_connections 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.799+0000 7f5ac046d700 1 -- 172.21.15.31:0/4118614506 wait complete. 2024-04-25T00:20:36.231 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.800+0000 7f5ac046d700 1 Processor -- start 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.800+0000 7f5ac046d700 1 -- start start 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.800+0000 7f5ac046d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab8107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.800+0000 7f5ac046d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab810bf40 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5abe209700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab8107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5abe209700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab8107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39646/0 (socket says 172.21.15.31:39646) 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5abe209700 1 -- 172.21.15.31:0/3299504787 learned_addr learned my addr 172.21.15.31:0/3299504787 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5abe209700 1 -- 172.21.15.31:0/3299504787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab8108410 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5abe209700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab8107ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4011a50 tx=0x7f5ab4011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ab4003a80 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5ab4003c20 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab8106600 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ab401a650 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.801+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab8106b40 con 0x7f5ab8108650 2024-04-25T00:20:36.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.802+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5ab4024440 con 0x7f5ab8108650 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.802+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab8063890 con 0x7f5ab8108650 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.802+0000 7f5aab7fe700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f5aa4039000 0x7f5aa403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.803+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ab404d4e0 con 0x7f5ab8108650 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.803+0000 7f5abda08700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f5aa4039000 0x7f5aa403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.803+0000 7f5abda08700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f5aa4039000 0x7f5aa403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006fd0 tx=0x7f5aac006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:29.808+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ab400e260 con 0x7f5ab8108650 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.101+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5ab8107330 con 0x7f5ab8108650 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.101+0000 7f5aab7fe700 1 -- 172.21.15.31:0/3299504787 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5ab404cf00 con 0x7f5ab8108650 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f5aa4039000 msgr2=0x7f5aa403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f5aa4039000 0x7f5aa403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006fd0 tx=0x7f5aac006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 msgr2=0x7f5ab8107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab8107ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4011a50 tx=0x7f5ab4011b30 comp rx=0 tx=0).stop 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 shutdown_connections 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f5aa4039000 0x7f5aa403b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 --2- 172.21.15.31:0/3299504787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab8108650 0x7f5ab8107ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 >> 172.21.15.31:0/3299504787 conn(0x7f5ab8102060 msgr2=0x7f5ab8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 shutdown_connections 2024-04-25T00:20:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:30.103+0000 7f5ac046d700 1 -- 172.21.15.31:0/3299504787 wait complete. 2024-04-25T00:20:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-04-25T00:20:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:36 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3583704277' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:20:37.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:37.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:20:37.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:20:37.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb688e0f700 1 Processor -- start 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb688e0f700 1 -- start start 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb688e0f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb684107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb688e0f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb684108590 con 0x7fb684107ba0 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb68259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb684107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb68259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb684107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44862/0 (socket says 172.21.15.31:44862) 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.673+0000 7fb68259c700 1 -- 172.21.15.31:0/1654450156 learned_addr learned my addr 172.21.15.31:0/1654450156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb68259c700 1 -- 172.21.15.31:0/1654450156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb684108da0 con 0x7fb684107ba0 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb68259c700 1 --2- 172.21.15.31:0/1654450156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb684107fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6740098a0 tx=0x7fb674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd9d7e7e6767c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb68159a700 1 -- 172.21.15.31:0/1654450156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb674004160 con 0x7fb684107ba0 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb68159a700 1 -- 172.21.15.31:0/1654450156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb674004300 con 0x7fb684107ba0 2024-04-25T00:20:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb68159a700 1 -- 172.21.15.31:0/1654450156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb674011a80 con 0x7fb684107ba0 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb688e0f700 1 -- 172.21.15.31:0/1654450156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 msgr2=0x7fb684107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.674+0000 7fb688e0f700 1 --2- 172.21.15.31:0/1654450156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb684107fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6740098a0 tx=0x7fb674009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.675+0000 7fb688e0f700 1 -- 172.21.15.31:0/1654450156 shutdown_connections 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.675+0000 7fb688e0f700 1 --2- 172.21.15.31:0/1654450156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb684107fc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.675+0000 7fb688e0f700 1 -- 172.21.15.31:0/1654450156 >> 172.21.15.31:0/1654450156 conn(0x7fb684103180 msgr2=0x7fb6841055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.675+0000 7fb688e0f700 1 -- 172.21.15.31:0/1654450156 shutdown_connections 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.675+0000 7fb688e0f700 1 -- 172.21.15.31:0/1654450156 wait complete. 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb688e0f700 1 Processor -- start 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb688e0f700 1 -- start start 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb688e0f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb68419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb688e0f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb68410c150 con 0x7fb684107ba0 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb68259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb68419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb68259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb68419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44878/0 (socket says 172.21.15.31:44878) 2024-04-25T00:20:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb68259c700 1 -- 172.21.15.31:0/3583704277 learned_addr learned my addr 172.21.15.31:0/3583704277 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb68259c700 1 -- 172.21.15.31:0/3583704277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68419c030 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb68259c700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb68419baf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb674000c00 tx=0x7fb674011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6740039f0 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68419c2c0 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb674003b90 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.676+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb67401a740 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.677+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68419efb0 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.677+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb67401aa40 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.677+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb684063890 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.678+0000 7fb6737fe700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7fb66c039000 0x7fb66c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.678+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb67404d3c0 con 0x7fb684107ba0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.678+0000 7fb681d9b700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7fb66c039000 0x7fb66c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.678+0000 7fb681d9b700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7fb66c039000 0x7fb66c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb678006fd0 tx=0x7fb678006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.683+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb674020240 con 0x7fb684107ba0 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.886+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb68419f1d0 con 0x7fb684107ba0 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.887+0000 7fb6737fe700 1 -- 172.21.15.31:0/3583704277 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fb68419f1d0 con 0x7fb684107ba0 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7fb66c039000 msgr2=0x7fb66c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7fb66c039000 0x7fb66c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb678006fd0 tx=0x7fb678006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 msgr2=0x7fb68419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb68419baf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb674000c00 tx=0x7fb674011b00 comp rx=0 tx=0).stop 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 shutdown_connections 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7fb66c039000 0x7fb66c03b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 --2- 172.21.15.31:0/3583704277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb684107ba0 0x7fb68419baf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 >> 172.21.15.31:0/3583704277 conn(0x7fb684103180 msgr2=0x7fb684103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 shutdown_connections 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:36.889+0000 7fb688e0f700 1 -- 172.21.15.31:0/3583704277 wait complete. 2024-04-25T00:20:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-04-25T00:20:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:38 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2830722960' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T00:20:38.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: ignoring --setuser ceph since I am not root 2024-04-25T00:20:38.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: ignoring --setgroup ceph since I am not root 2024-04-25T00:20:38.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:38.432+0000 7f9ff87e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:20:38.823 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:38.625+0000 7f9ff87e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:20:38.823 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:38.731+0000 7f9ff87e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:20:38.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.931+0000 7f22a8e76700 1 Processor -- start 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.931+0000 7f22a8e76700 1 -- start start 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.931+0000 7f22a8e76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a41068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.931+0000 7f22a8e76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f22a407bdb0 con 0x7f22a4106480 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a41068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a41068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44884/0 (socket says 172.21.15.31:44884) 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a259c700 1 -- 172.21.15.31:0/1780885689 learned_addr learned my addr 172.21.15.31:0/1780885689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a259c700 1 -- 172.21.15.31:0/1780885689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a4106de0 con 0x7f22a4106480 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a259c700 1 --2- 172.21.15.31:0/1780885689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a41068a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f229c009e80 tx=0x7f229c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e8f4460a59da24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a159a700 1 -- 172.21.15.31:0/1780885689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f229c017070 con 0x7f22a4106480 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a159a700 1 -- 172.21.15.31:0/1780885689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f229c00b8c0 con 0x7f22a4106480 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.932+0000 7f22a159a700 1 -- 172.21.15.31:0/1780885689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f229c004520 con 0x7f22a4106480 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 -- 172.21.15.31:0/1780885689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 msgr2=0x7f22a41068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 --2- 172.21.15.31:0/1780885689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a41068a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f229c009e80 tx=0x7f229c00b330 comp rx=0 tx=0).stop 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 -- 172.21.15.31:0/1780885689 shutdown_connections 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 --2- 172.21.15.31:0/1780885689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a41068a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 -- 172.21.15.31:0/1780885689 >> 172.21.15.31:0/1780885689 conn(0x7f22a4102020 msgr2=0x7f22a4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 -- 172.21.15.31:0/1780885689 shutdown_connections 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.933+0000 7f22a8e76700 1 -- 172.21.15.31:0/1780885689 wait complete. 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a8e76700 1 Processor -- start 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a8e76700 1 -- start start 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a8e76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a8e76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4107e10 con 0x7f22a4106480 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a419dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44892/0 (socket says 172.21.15.31:44892) 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a259c700 1 -- 172.21.15.31:0/2830722960 learned_addr learned my addr 172.21.15.31:0/2830722960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a259c700 1 -- 172.21.15.31:0/2830722960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a419e170 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.934+0000 7f22a259c700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a419dc30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b7b0 tx=0x7f229c004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.935+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f229c020020 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.935+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f229c00bcb0 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.935+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a419e400 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.935+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f229c017040 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.935+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a10f0 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.935+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f229c023020 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.936+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a4063890 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.936+0000 7f229b7fe700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2290039050 0x7f229003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.936+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f229c00f050 con 0x7f22a4106480 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.936+0000 7f22a1d9b700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2290039050 0x7f229003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.936+0000 7f22a1d9b700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2290039050 0x7f229003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f228c006fd0 tx=0x7f228c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:37.941+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f229c02a020 con 0x7f22a4106480 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.209+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f22a402dea0 con 0x7f22a4106480 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.260+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f229c007310 con 0x7f22a4106480 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.262+0000 7f229b7fe700 1 -- 172.21.15.31:0/2830722960 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f229c0129e0 con 0x7f22a4106480 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.263+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2290039050 msgr2=0x7f229003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.263+0000 7f22a8e76700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2290039050 0x7f229003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f228c006fd0 tx=0x7f228c006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.263+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 msgr2=0x7f22a419dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.263+0000 7f22a8e76700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a419dc30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b7b0 tx=0x7f229c004670 comp rx=0 tx=0).stop 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.263+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 shutdown_connections 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.263+0000 7f22a8e76700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2290039050 0x7f229003b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.264+0000 7f22a8e76700 1 --2- 172.21.15.31:0/2830722960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a4106480 0x7f22a419dc30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.264+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 >> 172.21.15.31:0/2830722960 conn(0x7f22a4102020 msgr2=0x7f22a41948a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.264+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 shutdown_connections 2024-04-25T00:20:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:38.264+0000 7f22a8e76700 1 -- 172.21.15.31:0/2830722960 wait complete. 2024-04-25T00:20:39.083 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:39 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:39.056+0000 7f9ff87e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:20:39.492 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:39 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2830722960' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T00:20:39.492 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:39 smithi031 ceph-mon[29167]: mgrmap e5: x(active, since 11s) 2024-04-25T00:20:39.493 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:39 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:39.147+0000 7f9ff87e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:20:40.164 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:20:40.164 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T00:20:40.164 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:20:40.164 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:20:40.164 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9c26d700 1 Processor -- start 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9c26d700 1 -- start start 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9c26d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b9407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9c26d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9407c8b0 con 0x7f2b9407de80 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9a009700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b9407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9a009700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b9407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44922/0 (socket says 172.21.15.31:44922) 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9a009700 1 -- 172.21.15.31:0/4216052998 learned_addr learned my addr 172.21.15.31:0/4216052998 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.286+0000 7f2b9a009700 1 -- 172.21.15.31:0/4216052998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9407ca30 con 0x7f2b9407de80 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.287+0000 7f2b9a009700 1 --2- 172.21.15.31:0/4216052998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b9407c2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b880098a0 tx=0x7f2b88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199cc4af9ff746cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.287+0000 7f2b99007700 1 -- 172.21.15.31:0/4216052998 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b88004160 con 0x7f2b9407de80 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.287+0000 7f2b99007700 1 -- 172.21.15.31:0/4216052998 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2b88004300 con 0x7f2b9407de80 2024-04-25T00:20:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.287+0000 7f2b99007700 1 -- 172.21.15.31:0/4216052998 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b88011a80 con 0x7f2b9407de80 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.287+0000 7f2b9c26d700 1 -- 172.21.15.31:0/4216052998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 msgr2=0x7f2b9407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.287+0000 7f2b9c26d700 1 --2- 172.21.15.31:0/4216052998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b9407c2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b880098a0 tx=0x7f2b88009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 -- 172.21.15.31:0/4216052998 shutdown_connections 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 --2- 172.21.15.31:0/4216052998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b9407c2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 -- 172.21.15.31:0/4216052998 >> 172.21.15.31:0/4216052998 conn(0x7f2b94103110 msgr2=0x7f2b94105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 -- 172.21.15.31:0/4216052998 shutdown_connections 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 -- 172.21.15.31:0/4216052998 wait complete. 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 Processor -- start 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 -- start start 2024-04-25T00:20:40.166 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.288+0000 7f2b9c26d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b941a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9c26d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2b94109ce0 con 0x7f2b9407de80 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9a009700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b941a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9a009700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b941a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44924/0 (socket says 172.21.15.31:44924) 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9a009700 1 -- 172.21.15.31:0/3604142882 learned_addr learned my addr 172.21.15.31:0/3604142882 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9a009700 1 -- 172.21.15.31:0/3604142882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b941a25f0 con 0x7f2b9407de80 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9a009700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b941a20b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006af0 tx=0x7f2b88011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b880039e0 con 0x7f2b9407de80 2024-04-25T00:20:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2b88003b80 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b941a2880 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b880196a0 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.289+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b941a5570 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.290+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f2b8801ab50 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.290+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b94063890 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.290+0000 7f2b877fe700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 0x7f2b8003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.291+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b8804d510 con 0x7f2b9407de80 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.291+0000 7f2b99808700 1 -- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 msgr2=0x7f2b8003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:40.168 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.291+0000 7f2b99808700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 0x7f2b8003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.295+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2b8800b3e0 con 0x7f2b9407de80 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.492+0000 7f2b99808700 1 -- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 msgr2=0x7f2b8003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.492+0000 7f2b99808700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 0x7f2b8003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.602+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2b941a5740 con 0x7f2b9407de80 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.605+0000 7f2b877fe700 1 -- 172.21.15.31:0/3604142882 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f2b941a5740 con 0x7f2b9407de80 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 msgr2=0x7f2b8003b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 0x7f2b8003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 msgr2=0x7f2b941a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b941a20b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006af0 tx=0x7f2b88011c40 comp rx=0 tx=0).stop 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 shutdown_connections 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7f2b80039050 0x7f2b8003b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 --2- 172.21.15.31:0/3604142882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b9407de80 0x7f2b941a20b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 >> 172.21.15.31:0/3604142882 conn(0x7f2b94103110 msgr2=0x7f2b94105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 shutdown_connections 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:39.608+0000 7f2b9c26d700 1 -- 172.21.15.31:0/3604142882 wait complete. 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-25T00:20:40.170 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-04-25T00:20:40.420 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:40 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3604142882' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:20:40.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:40.420+0000 7f9ff87e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:20:40.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:40.599+0000 7f9ff87e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:20:41.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:40.795+0000 7f9ff87e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:20:41.404 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:41.094+0000 7f9ff87e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:20:41.404 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:41.189+0000 7f9ff87e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:20:41.405 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:41.404+0000 7f9ff87e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:20:41.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:41.603+0000 7f9ff87e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:20:42.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:42.228+0000 7f9ff87e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:20:42.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:42.666+0000 7f9ff87e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:20:42.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:42.765+0000 7f9ff87e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:20:42.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:42.972+0000 7f9ff87e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:20:43.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:43.519+0000 7f9ff87e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:20:43.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:43.613+0000 7f9ff87e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:20:44.222 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:43.863+0000 7f9ff87e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:20:44.222 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:44.002+0000 7f9ff87e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:20:44.222 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:44.100+0000 7f9ff87e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:20:44.223 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:44.221+0000 7f9ff87e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:20:44.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:44.316+0000 7f9ff87e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:20:45.433 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:45.158+0000 7f9ff87e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:20:45.433 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:20:45.380+0000 7f9ff87e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:20:45.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: Active manager daemon x restarted 2024-04-25T00:20:45.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: Activating manager daemon x 2024-04-25T00:20:45.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T00:20:45.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: mgrmap e6: x(active, starting, since 0.00364735s) 2024-04-25T00:20:45.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:20:45.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: Manager daemon x is now available 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:45.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:45 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:20:46.701 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:46 smithi031 ceph-mon[29167]: Found migration_current of "None". Setting to last migration. 2024-04-25T00:20:46.701 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:46 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:20:46.701 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:46 smithi031 ceph-mon[29167]: mgrmap e7: x(active, since 1.00584s) 2024-04-25T00:20:46.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:20:46.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T00:20:46.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:20:46.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:20:46.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.616+0000 7ff7b59f6700 1 Processor -- start 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.616+0000 7ff7b59f6700 1 -- start start 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.616+0000 7ff7b59f6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0106e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.616+0000 7ff7b59f6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0107440 con 0x7ff7b0106a50 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0106e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44940/0 (socket says 172.21.15.31:44940) 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7aeffd700 1 -- 172.21.15.31:0/3121885039 learned_addr learned my addr 172.21.15.31:0/3121885039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7aeffd700 1 -- 172.21.15.31:0/3121885039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b0107c50 con 0x7ff7b0106a50 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7aeffd700 1 --2- 172.21.15.31:0/3121885039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0106e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098a0 tx=0x7ff7a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ee949262ffaa09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7adffb700 1 -- 172.21.15.31:0/3121885039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a4004160 con 0x7ff7b0106a50 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7adffb700 1 -- 172.21.15.31:0/3121885039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff7a4004300 con 0x7ff7b0106a50 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.617+0000 7ff7adffb700 1 -- 172.21.15.31:0/3121885039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a4011a80 con 0x7ff7b0106a50 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 -- 172.21.15.31:0/3121885039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 msgr2=0x7ff7b0106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/3121885039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0106e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098a0 tx=0x7ff7a4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:46.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 -- 172.21.15.31:0/3121885039 shutdown_connections 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/3121885039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0106e70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 -- 172.21.15.31:0/3121885039 >> 172.21.15.31:0/3121885039 conn(0x7ff7b0101ff0 msgr2=0x7ff7b0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 -- 172.21.15.31:0/3121885039 shutdown_connections 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.618+0000 7ff7b59f6700 1 -- 172.21.15.31:0/3121885039 wait complete. 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7b59f6700 1 Processor -- start 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7b59f6700 1 -- start start 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7b59f6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0191bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7b59f6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010b000 con 0x7ff7b0106a50 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0191bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0191bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44956/0 (socket says 172.21.15.31:44956) 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7aeffd700 1 -- 172.21.15.31:0/2659096361 learned_addr learned my addr 172.21.15.31:0/2659096361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.619+0000 7ff7aeffd700 1 -- 172.21.15.31:0/2659096361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b0192100 con 0x7ff7b0106a50 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.620+0000 7ff7aeffd700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0191bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4000c00 tx=0x7ff7a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.620+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a40039f0 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.620+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff7a4003b90 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.620+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b01902f0 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.620+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a401a740 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.620+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b0190830 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.621+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff7a401aa40 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.621+0000 7ff7b49f4700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.621+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 --> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff79803bc20 con 0x7ff798039050 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.621+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff7a404d3d0 con 0x7ff7b0106a50 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.621+0000 7ff7ae7fc700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 msgr2=0x7ff79803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.621+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.822+0000 7ff7ae7fc700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 msgr2=0x7ff79803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:40.822+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:41.223+0000 7ff7ae7fc700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 msgr2=0x7ff79803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:41.223+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:42.024+0000 7ff7ae7fc700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 msgr2=0x7ff79803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:42.024+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:43.625+0000 7ff7ae7fc700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 msgr2=0x7ff79803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1911777180 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:43.625+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:45.384+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7ff7a402d910 con 0x7ff7b0106a50 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:45.384+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 msgr2=0x7ff79803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:45.384+0000 7ff7b49f4700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.385+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7ff7a4029cf0 con 0x7ff7b0106a50 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.385+0000 7ff7b49f4700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7ff79803c980 0x7ff79803ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.385+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff7a40068a0 con 0x7ff79803c980 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.386+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7ff79803c980 0x7ff79803ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:46.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.386+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7ff79803c980 0x7ff79803ed70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0003a10 tx=0x7ff7a0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.387+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff7a40068a0 con 0x7ff79803c980 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.393+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff7b0063890 con 0x7ff79803c980 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b49f4700 1 -- 172.21.15.31:0/2659096361 <== mgr.14118 v2:172.21.15.31:6800/3443312222 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff7b0063890 con 0x7ff79803c980 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7ff79803c980 msgr2=0x7ff79803ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7ff79803c980 0x7ff79803ed70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0003a10 tx=0x7ff7a0009290 comp rx=0 tx=0).stop 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 msgr2=0x7ff7b0191bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0191bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4000c00 tx=0x7ff7a4011b00 comp rx=0 tx=0).stop 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 shutdown_connections 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7ff79803c980 0x7ff79803ed70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0106a50 0x7ff7b0191bc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 --2- 172.21.15.31:0/2659096361 >> [v2:172.21.15.31:6800/1911777180,v1:172.21.15.31:6801/1911777180] conn(0x7ff798039050 0x7ff79803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 >> 172.21.15.31:0/2659096361 conn(0x7ff7b0101ff0 msgr2=0x7ff7b0102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 shutdown_connections 2024-04-25T00:20:46.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:46.394+0000 7ff7b59f6700 1 -- 172.21.15.31:0/2659096361 wait complete. 2024-04-25T00:20:46.984 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-04-25T00:20:46.984 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-04-25T00:20:47.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:20:47.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:20:47.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:47.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:47.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: [25/Apr/2024:00:20:47] ENGINE Bus STARTING 2024-04-25T00:20:47.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: [25/Apr/2024:00:20:47] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T00:20:47.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:47 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:48.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafca7bf700 1 Processor -- start 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafca7bf700 1 -- start start 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafca7bf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc4107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafca7bf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4108590 con 0x7fafc4107ba0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc4107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc4107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40470/0 (socket says 172.21.15.31:40470) 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.348+0000 7fafc3fff700 1 -- 172.21.15.31:0/2875529141 learned_addr learned my addr 172.21.15.31:0/2875529141 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafc3fff700 1 -- 172.21.15.31:0/2875529141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc4108da0 con 0x7fafc4107ba0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafc3fff700 1 --2- 172.21.15.31:0/2875529141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc4107fc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fafb40098a0 tx=0x7fafb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83066eea2aa01f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafc2ffd700 1 -- 172.21.15.31:0/2875529141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb4004160 con 0x7fafc4107ba0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafc2ffd700 1 -- 172.21.15.31:0/2875529141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fafb4004300 con 0x7fafc4107ba0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafc2ffd700 1 -- 172.21.15.31:0/2875529141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb4011a80 con 0x7fafc4107ba0 2024-04-25T00:20:48.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafca7bf700 1 -- 172.21.15.31:0/2875529141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 msgr2=0x7fafc4107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafca7bf700 1 --2- 172.21.15.31:0/2875529141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc4107fc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fafb40098a0 tx=0x7fafb4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafca7bf700 1 -- 172.21.15.31:0/2875529141 shutdown_connections 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafca7bf700 1 --2- 172.21.15.31:0/2875529141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc4107fc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.349+0000 7fafca7bf700 1 -- 172.21.15.31:0/2875529141 >> 172.21.15.31:0/2875529141 conn(0x7fafc4103180 msgr2=0x7fafc41055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.350+0000 7fafca7bf700 1 -- 172.21.15.31:0/2875529141 shutdown_connections 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.350+0000 7fafca7bf700 1 -- 172.21.15.31:0/2875529141 wait complete. 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.350+0000 7fafca7bf700 1 Processor -- start 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.350+0000 7fafca7bf700 1 -- start start 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafca7bf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafca7bf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410c150 con 0x7fafc4107ba0 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40476/0 (socket says 172.21.15.31:40476) 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc3fff700 1 -- 172.21.15.31:0/1621035742 learned_addr learned my addr 172.21.15.31:0/1621035742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc3fff700 1 -- 172.21.15.31:0/1621035742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc419c030 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc3fff700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc419baf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c00 tx=0x7fafb4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb40039f0 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fafb4003b90 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc419c2c0 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb401a740 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.351+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc419efb0 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.352+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fafb401aa40 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.352+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc4063890 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.352+0000 7fafc17fa700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fafac038f30 0x7fafac03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.352+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fafb404d3f0 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.352+0000 7fafc37fe700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fafac038f30 0x7fafac03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.353+0000 7fafc37fe700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fafac038f30 0x7fafac03b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006fd0 tx=0x7fafb8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.357+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fafb401ade0 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.562+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fafb402d910 con 0x7fafc4107ba0 2024-04-25T00:20:48.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.584+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fafc4104380 con 0x7fafac038f30 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.591+0000 7fafc17fa700 1 -- 172.21.15.31:0/1621035742 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fafc4104380 con 0x7fafac038f30 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fafac038f30 msgr2=0x7fafac03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fafac038f30 0x7fafac03b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006fd0 tx=0x7fafb8006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 msgr2=0x7fafc419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc419baf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c00 tx=0x7fafb4011b00 comp rx=0 tx=0).stop 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 shutdown_connections 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fafac038f30 0x7fafac03b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 --2- 172.21.15.31:0/1621035742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafc4107ba0 0x7fafc419baf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 >> 172.21.15.31:0/1621035742 conn(0x7fafc4103180 msgr2=0x7fafc4103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 shutdown_connections 2024-04-25T00:20:48.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:47.594+0000 7fafca7bf700 1 -- 172.21.15.31:0/1621035742 wait complete. 2024-04-25T00:20:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:48 smithi031 ceph-mon[29167]: [25/Apr/2024:00:20:47] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T00:20:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:48 smithi031 ceph-mon[29167]: [25/Apr/2024:00:20:47] ENGINE Bus STARTED 2024-04-25T00:20:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:48 smithi031 ceph-mon[29167]: mgrmap e8: x(active, since 2s) 2024-04-25T00:20:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:48 smithi031 ceph-mon[29167]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:48 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:48 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:49.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T00:20:49.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.624+0000 7fee212ef700 1 Processor -- start 2024-04-25T00:20:49.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.624+0000 7fee212ef700 1 -- start start 2024-04-25T00:20:49.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.624+0000 7fee212ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.624+0000 7fee212ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c106830 con 0x7fee1c105e40 2024-04-25T00:20:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.624+0000 7fee1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40484/0 (socket says 172.21.15.31:40484) 2024-04-25T00:20:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1affd700 1 -- 172.21.15.31:0/2544819854 learned_addr learned my addr 172.21.15.31:0/2544819854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1affd700 1 -- 172.21.15.31:0/2544819854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c108dd0 con 0x7fee1c105e40 2024-04-25T00:20:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1affd700 1 --2- 172.21.15.31:0/2544819854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c106260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0098a0 tx=0x7fee0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1ed73d3f0f1e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1a7fc700 1 -- 172.21.15.31:0/2544819854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee0c004160 con 0x7fee1c105e40 2024-04-25T00:20:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1a7fc700 1 -- 172.21.15.31:0/2544819854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fee0c004300 con 0x7fee1c105e40 2024-04-25T00:20:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.625+0000 7fee1a7fc700 1 -- 172.21.15.31:0/2544819854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee0c011a80 con 0x7fee1c105e40 2024-04-25T00:20:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 -- 172.21.15.31:0/2544819854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 msgr2=0x7fee1c106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 --2- 172.21.15.31:0/2544819854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c106260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0098a0 tx=0x7fee0c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 -- 172.21.15.31:0/2544819854 shutdown_connections 2024-04-25T00:20:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 --2- 172.21.15.31:0/2544819854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c106260 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 -- 172.21.15.31:0/2544819854 >> 172.21.15.31:0/2544819854 conn(0x7fee1c1017f0 msgr2=0x7fee1c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 -- 172.21.15.31:0/2544819854 shutdown_connections 2024-04-25T00:20:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 -- 172.21.15.31:0/2544819854 wait complete. 2024-04-25T00:20:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 Processor -- start 2024-04-25T00:20:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.626+0000 7fee212ef700 1 -- start start 2024-04-25T00:20:49.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee212ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c108650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:49.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee212ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c10c180 con 0x7fee1c105e40 2024-04-25T00:20:49.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c108650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:49.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c108650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40492/0 (socket says 172.21.15.31:40492) 2024-04-25T00:20:49.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee1affd700 1 -- 172.21.15.31:0/280496571 learned_addr learned my addr 172.21.15.31:0/280496571 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:49.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee1affd700 1 -- 172.21.15.31:0/280496571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c108b90 con 0x7fee1c105e40 2024-04-25T00:20:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee1affd700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c108650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fee0c006af0 tx=0x7fee0c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee0c0039e0 con 0x7fee1c105e40 2024-04-25T00:20:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fee0c003b80 con 0x7fee1c105e40 2024-04-25T00:20:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c106d80 con 0x7fee1c105e40 2024-04-25T00:20:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee0c0196a0 con 0x7fee1c105e40 2024-04-25T00:20:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.627+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c1072c0 con 0x7fee1c105e40 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.628+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fee0c0199a0 con 0x7fee1c105e40 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.628+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1c063890 con 0x7fee1c105e40 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.628+0000 7fee18ff9700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fee08039000 0x7fee0803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.629+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fee0c04d5e0 con 0x7fee1c105e40 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.629+0000 7fee13fff700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fee08039000 0x7fee0803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.629+0000 7fee13fff700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fee08039000 0x7fee0803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee04006fd0 tx=0x7fee04006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.634+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee0c019d90 con 0x7fee1c105e40 2024-04-25T00:20:49.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.841+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fee1c107b20 con 0x7fee08039000 2024-04-25T00:20:49.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.841+0000 7fee18ff9700 1 -- 172.21.15.31:0/280496571 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fee1c107b20 con 0x7fee08039000 2024-04-25T00:20:49.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fee08039000 msgr2=0x7fee0803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:49.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fee08039000 0x7fee0803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee04006fd0 tx=0x7fee04006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:49.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 msgr2=0x7fee1c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:49.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c108650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fee0c006af0 tx=0x7fee0c011c40 comp rx=0 tx=0).stop 2024-04-25T00:20:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 shutdown_connections 2024-04-25T00:20:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fee08039000 0x7fee0803b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 --2- 172.21.15.31:0/280496571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee1c105e40 0x7fee1c108650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 >> 172.21.15.31:0/280496571 conn(0x7fee1c1017f0 msgr2=0x7fee1c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 shutdown_connections 2024-04-25T00:20:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:48.843+0000 7fee212ef700 1 -- 172.21.15.31:0/280496571 wait complete. 2024-04-25T00:20:49.411 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-04-25T00:20:49.682 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:49 smithi031 ceph-mon[29167]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:50.819 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-mon[29167]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:50.819 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-mon[29167]: Generating ssh key... 2024-04-25T00:20:51.128 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: Generating public/private rsa key pair. 2024-04-25T00:20:51.128 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: Your identification has been saved in /tmp/tmpm4nb92nw/key. 2024-04-25T00:20:51.128 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: Your public key has been saved in /tmp/tmpm4nb92nw/key.pub. 2024-04-25T00:20:51.128 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: The key fingerprint is: 2024-04-25T00:20:51.128 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: SHA256:jLWkoDXhKZbwH8jcrHp4iGnZqfjtG0tXvusl+2ipd0Q ceph-6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: The key's randomart image is: 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: +---[RSA 3072]----+ 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: |. . | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: | = * o | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: | O X o | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: | . * + * .E | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: | o . o S. | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: |.=o . o . | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: |*oooo . oo. | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: |oo.o + +*. | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: |.o..=..=*+. | 2024-04-25T00:20:51.129 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:20:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: +----[SHA256]-----+ 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.794+0000 7fd962351700 1 Processor -- start 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.794+0000 7fd962351700 1 -- start start 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.794+0000 7fd962351700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.794+0000 7fd962351700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c07c870 con 0x7fd95c07de40 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40496/0 (socket says 172.21.15.31:40496) 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95bfff700 1 -- 172.21.15.31:0/56588277 learned_addr learned my addr 172.21.15.31:0/56588277 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95bfff700 1 -- 172.21.15.31:0/56588277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c07c9f0 con 0x7fd95c07de40 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95bfff700 1 --2- 172.21.15.31:0/56588277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c07c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0098a0 tx=0x7fd94c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a021560f58af3ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95affd700 1 -- 172.21.15.31:0/56588277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd94c004160 con 0x7fd95c07de40 2024-04-25T00:20:51.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95affd700 1 -- 172.21.15.31:0/56588277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd94c004300 con 0x7fd95c07de40 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.795+0000 7fd95affd700 1 -- 172.21.15.31:0/56588277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd94c011a80 con 0x7fd95c07de40 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 -- 172.21.15.31:0/56588277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 msgr2=0x7fd95c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 --2- 172.21.15.31:0/56588277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c07c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0098a0 tx=0x7fd94c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 -- 172.21.15.31:0/56588277 shutdown_connections 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 --2- 172.21.15.31:0/56588277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c07c2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 -- 172.21.15.31:0/56588277 >> 172.21.15.31:0/56588277 conn(0x7fd95c103180 msgr2=0x7fd95c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 -- 172.21.15.31:0/56588277 shutdown_connections 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 -- 172.21.15.31:0/56588277 wait complete. 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.796+0000 7fd962351700 1 Processor -- start 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd962351700 1 -- start start 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd962351700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd962351700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c10e150 con 0x7fd95c07de40 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd95bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd95bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c19dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40502/0 (socket says 172.21.15.31:40502) 2024-04-25T00:20:51.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd95bfff700 1 -- 172.21.15.31:0/3739659927 learned_addr learned my addr 172.21.15.31:0/3739659927 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd95bfff700 1 -- 172.21.15.31:0/3739659927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c19e230 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.797+0000 7fd95bfff700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c19dcf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006af0 tx=0x7fd94c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.798+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd94c0039e0 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.798+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd94c003b80 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.798+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95c19e4c0 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.798+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd94c0196a0 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.798+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95c1a11a0 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.799+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd94c0199a0 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.799+0000 7fd9597fa700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fd948039000 0x7fd94803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.799+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd94c04d5e0 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.799+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95c063890 con 0x7fd95c07de40 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.799+0000 7fd95b7fe700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fd948039000 0x7fd94803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.799+0000 7fd95b7fe700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fd948039000 0x7fd94803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd944006fd0 tx=0x7fd944006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:49.804+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd94c020240 con 0x7fd95c07de40 2024-04-25T00:20:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.010+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd95c104380 con 0x7fd948039000 2024-04-25T00:20:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.824+0000 7fd9597fa700 1 -- 172.21.15.31:0/3739659927 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd95c104380 con 0x7fd948039000 2024-04-25T00:20:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.826+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fd948039000 msgr2=0x7fd94803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.826+0000 7fd962351700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fd948039000 0x7fd94803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd944006fd0 tx=0x7fd944006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.826+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 msgr2=0x7fd95c19dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.826+0000 7fd962351700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c19dcf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006af0 tx=0x7fd94c011c40 comp rx=0 tx=0).stop 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.827+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 shutdown_connections 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.827+0000 7fd962351700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fd948039000 0x7fd94803b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.827+0000 7fd962351700 1 --2- 172.21.15.31:0/3739659927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95c07de40 0x7fd95c19dcf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.827+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 >> 172.21.15.31:0/3739659927 conn(0x7fd95c103180 msgr2=0x7fd95c103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.827+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 shutdown_connections 2024-04-25T00:20:51.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:50.827+0000 7fd962351700 1 -- 172.21.15.31:0/3739659927 wait complete. 2024-04-25T00:20:52.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:51 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:52.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:51 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6622acb700 1 Processor -- start 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6622acb700 1 -- start start 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6622acb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6622acb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f661c106850 con 0x7f661c105e60 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6620867700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6620867700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40514/0 (socket says 172.21.15.31:40514) 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.883+0000 7f6620867700 1 -- 172.21.15.31:0/1984998810 learned_addr learned my addr 172.21.15.31:0/1984998810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.884+0000 7f6620867700 1 -- 172.21.15.31:0/1984998810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c108df0 con 0x7f661c105e60 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.884+0000 7f6620867700 1 --2- 172.21.15.31:0/1984998810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c106280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f660c0098a0 tx=0x7f660c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6316aad1409f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.884+0000 7f661b7fe700 1 -- 172.21.15.31:0/1984998810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f660c004160 con 0x7f661c105e60 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.884+0000 7f661b7fe700 1 -- 172.21.15.31:0/1984998810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f660c004300 con 0x7f661c105e60 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.884+0000 7f661b7fe700 1 -- 172.21.15.31:0/1984998810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f660c011a80 con 0x7f661c105e60 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 -- 172.21.15.31:0/1984998810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 msgr2=0x7f661c106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 --2- 172.21.15.31:0/1984998810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c106280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f660c0098a0 tx=0x7f660c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 -- 172.21.15.31:0/1984998810 shutdown_connections 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 --2- 172.21.15.31:0/1984998810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c106280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 -- 172.21.15.31:0/1984998810 >> 172.21.15.31:0/1984998810 conn(0x7f661c101830 msgr2=0x7f661c103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 -- 172.21.15.31:0/1984998810 shutdown_connections 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 -- 172.21.15.31:0/1984998810 wait complete. 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 Processor -- start 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.885+0000 7f6622acb700 1 -- start start 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6622acb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6622acb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f661c10c1a0 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6620867700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6620867700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40528/0 (socket says 172.21.15.31:40528) 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6620867700 1 -- 172.21.15.31:0/2745847379 learned_addr learned my addr 172.21.15.31:0/2745847379 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6620867700 1 -- 172.21.15.31:0/2745847379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c19c010 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6620867700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c19bad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f660c006af0 tx=0x7f660c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f660c0039e0 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f660c003b80 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c19c2a0 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.886+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f660c0196a0 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.887+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c19ef90 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.887+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f660c01ab50 con 0x7f661c105e60 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.888+0000 7f6619ffb700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f6604039000 0x7f660403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.888+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661c063890 con 0x7f661c105e60 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.888+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f660c04d590 con 0x7f661c105e60 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.888+0000 7f661bfff700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f6604039000 0x7f660403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.888+0000 7f661bfff700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f6604039000 0x7f660403b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6610006fd0 tx=0x7f6610006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:51.893+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f660c00b3e0 con 0x7f661c105e60 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.102+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f661c1029c0 con 0x7f6604039000 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.103+0000 7f6619ffb700 1 -- 172.21.15.31:0/2745847379 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f661c1029c0 con 0x7f6604039000 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f6604039000 msgr2=0x7f660403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f6604039000 0x7f660403b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6610006fd0 tx=0x7f6610006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 msgr2=0x7f661c19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c19bad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f660c006af0 tx=0x7f660c011c40 comp rx=0 tx=0).stop 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 shutdown_connections 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f6604039000 0x7f660403b4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 --2- 172.21.15.31:0/2745847379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f661c105e60 0x7f661c19bad0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 >> 172.21.15.31:0/2745847379 conn(0x7f661c101830 msgr2=0x7f661c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 shutdown_connections 2024-04-25T00:20:52.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:52.105+0000 7f6622acb700 1 -- 172.21.15.31:0/2745847379 wait complete. 2024-04-25T00:20:52.696 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:20:52.696 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T00:20:52.696 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-04-25T00:20:52.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:52 smithi031 ceph-mon[29167]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:54 smithi031 ceph-mon[29167]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:54 smithi031 ceph-mon[29167]: Deploying cephadm binary to smithi031 2024-04-25T00:20:58.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-04-25T00:20:58.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.150+0000 7f011df96700 1 Processor -- start 2024-04-25T00:20:58.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.150+0000 7f011df96700 1 -- start start 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.150+0000 7f011df96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011810aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.150+0000 7f011df96700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f011807be10 con 0x7f0118108630 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011810aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011810aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36278/0 (socket says 172.21.15.31:36278) 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01177fe700 1 -- 172.21.15.31:0/2938283278 learned_addr learned my addr 172.21.15.31:0/2938283278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01177fe700 1 -- 172.21.15.31:0/2938283278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011810af60 con 0x7f0118108630 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01177fe700 1 --2- 172.21.15.31:0/2938283278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011810aa20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01100098a0 tx=0x7f0110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6075b7d01ff06b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01167fc700 1 -- 172.21.15.31:0/2938283278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0110004160 con 0x7f0118108630 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01167fc700 1 -- 172.21.15.31:0/2938283278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0110004300 con 0x7f0118108630 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.151+0000 7f01167fc700 1 -- 172.21.15.31:0/2938283278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0110011a80 con 0x7f0118108630 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 -- 172.21.15.31:0/2938283278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 msgr2=0x7f011810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 --2- 172.21.15.31:0/2938283278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011810aa20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01100098a0 tx=0x7f0110009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 -- 172.21.15.31:0/2938283278 shutdown_connections 2024-04-25T00:20:58.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 --2- 172.21.15.31:0/2938283278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011810aa20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 -- 172.21.15.31:0/2938283278 >> 172.21.15.31:0/2938283278 conn(0x7f0118102020 msgr2=0x7f0118104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 -- 172.21.15.31:0/2938283278 shutdown_connections 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.152+0000 7f011df96700 1 -- 172.21.15.31:0/2938283278 wait complete. 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f011df96700 1 Processor -- start 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f011df96700 1 -- start start 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f011df96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f011df96700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f011810bf20 con 0x7f0118108630 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f01177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f01177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36282/0 (socket says 172.21.15.31:36282) 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f01177fe700 1 -- 172.21.15.31:0/1517524517 learned_addr learned my addr 172.21.15.31:0/1517524517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.153+0000 7f01177fe700 1 -- 172.21.15.31:0/1517524517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011819bfd0 con 0x7f0118108630 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.154+0000 7f01177fe700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011819ba90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0110006af0 tx=0x7f0110011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.154+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01100039e0 con 0x7f0118108630 2024-04-25T00:20:58.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.154+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0110003b80 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.154+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011819c260 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.154+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01100196a0 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.154+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011819ef50 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.155+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f01100199a0 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.155+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0118063890 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.155+0000 7f0114ff9700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f0104039000 0x7f010403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.155+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f011004d630 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.155+0000 7f0116ffd700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f0104039000 0x7f010403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.156+0000 7f0116ffd700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f0104039000 0x7f010403b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0100006fd0 tx=0x7f0100006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.160+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f011002d9c0 con 0x7f0118108630 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:53.372+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) -- 0x7f0118103220 con 0x7f0104039000 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.579+0000 7f0114ff9700 1 -- 172.21.15.31:0/1517524517 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0118103220 con 0x7f0104039000 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f0104039000 msgr2=0x7f010403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:58.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f0104039000 0x7f010403b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0100006fd0 tx=0x7f0100006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 msgr2=0x7f011819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011819ba90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0110006af0 tx=0x7f0110011c40 comp rx=0 tx=0).stop 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 shutdown_connections 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f0104039000 0x7f010403b4c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 --2- 172.21.15.31:0/1517524517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0118108630 0x7f011819ba90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 >> 172.21.15.31:0/1517524517 conn(0x7f0118102020 msgr2=0x7f0118102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 shutdown_connections 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:57.582+0000 7f011df96700 1 -- 172.21.15.31:0/1517524517 wait complete. 2024-04-25T00:20:58.136 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-04-25T00:20:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:58 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:20:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:58 smithi031 ceph-mon[29167]: Added host smithi031 2024-04-25T00:20:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:58 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:59.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T00:20:59.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.606+0000 7f888fca8700 1 Processor -- start 2024-04-25T00:20:59.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.606+0000 7f888fca8700 1 -- start start 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888fca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f8888107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888fca8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8888108590 con 0x7f8888107ba0 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888da44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f8888107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888da44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f8888107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36284/0 (socket says 172.21.15.31:36284) 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888da44700 1 -- 172.21.15.31:0/776953661 learned_addr learned my addr 172.21.15.31:0/776953661 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888da44700 1 -- 172.21.15.31:0/776953661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8888108da0 con 0x7f8888107ba0 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.607+0000 7f888da44700 1 --2- 172.21.15.31:0/776953661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f8888107fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88840098a0 tx=0x7f8884009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0c1b10035b6d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888ca42700 1 -- 172.21.15.31:0/776953661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8884004160 con 0x7f8888107ba0 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888ca42700 1 -- 172.21.15.31:0/776953661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8884004300 con 0x7f8888107ba0 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888ca42700 1 -- 172.21.15.31:0/776953661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8884011a80 con 0x7f8888107ba0 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 -- 172.21.15.31:0/776953661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 msgr2=0x7f8888107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 --2- 172.21.15.31:0/776953661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f8888107fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88840098a0 tx=0x7f8884009bb0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 -- 172.21.15.31:0/776953661 shutdown_connections 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 --2- 172.21.15.31:0/776953661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f8888107fc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 -- 172.21.15.31:0/776953661 >> 172.21.15.31:0/776953661 conn(0x7f8888103180 msgr2=0x7f88881055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 -- 172.21.15.31:0/776953661 shutdown_connections 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.608+0000 7f888fca8700 1 -- 172.21.15.31:0/776953661 wait complete. 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888fca8700 1 Processor -- start 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888fca8700 1 -- start start 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888fca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f888819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888fca8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f888810c150 con 0x7f8888107ba0 2024-04-25T00:20:59.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888da44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f888819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888da44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f888819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36296/0 (socket says 172.21.15.31:36296) 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888da44700 1 -- 172.21.15.31:0/1657945668 learned_addr learned my addr 172.21.15.31:0/1657945668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888da44700 1 -- 172.21.15.31:0/1657945668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888819c030 con 0x7f8888107ba0 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.609+0000 7f888da44700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f888819baf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.610+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88840039f0 con 0x7f8888107ba0 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.610+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8884003b90 con 0x7f8888107ba0 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.610+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f888401a740 con 0x7f8888107ba0 2024-04-25T00:20:59.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.610+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888819c2c0 con 0x7f8888107ba0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.610+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888819efb0 con 0x7f8888107ba0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.611+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8884024af0 con 0x7f8888107ba0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.611+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8888063890 con 0x7f8888107ba0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.611+0000 7f887effd700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f88740390a0 0x7f887403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.611+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f888404d380 con 0x7f8888107ba0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.611+0000 7f888d243700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f88740390a0 0x7f887403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.611+0000 7f888d243700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f88740390a0 0x7f887403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8878006fd0 tx=0x7f8878006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.616+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8884020db0 con 0x7f8888107ba0 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.839+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8888104380 con 0x7f88740390a0 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.845+0000 7f887effd700 1 -- 172.21.15.31:0/1657945668 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8888104380 con 0x7f88740390a0 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f88740390a0 msgr2=0x7f887403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f88740390a0 0x7f887403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8878006fd0 tx=0x7f8878006e40 comp rx=0 tx=0).stop 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 msgr2=0x7f888819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f888819baf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884011b00 comp rx=0 tx=0).stop 2024-04-25T00:20:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 shutdown_connections 2024-04-25T00:20:59.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f88740390a0 0x7f887403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 --2- 172.21.15.31:0/1657945668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8888107ba0 0x7f888819baf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 >> 172.21.15.31:0/1657945668 conn(0x7f8888103180 msgr2=0x7f8888103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:59.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 shutdown_connections 2024-04-25T00:20:59.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:58.847+0000 7f888fca8700 1 -- 172.21.15.31:0/1657945668 wait complete. 2024-04-25T00:20:59.435 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-04-25T00:21:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:59 smithi031 ceph-mon[29167]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:59 smithi031 ceph-mon[29167]: Saving service mon spec with placement count:5 2024-04-25T00:21:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:20:59 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:21:01.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:01 smithi031 ceph-mon[29167]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:01.395 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:01 smithi031 ceph-mon[29167]: Saving service mgr spec with placement count:2 2024-04-25T00:21:01.395 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:01 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:21:01.395 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:01 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:21:01.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f453bf7d700 1 Processor -- start 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f453bf7d700 1 -- start start 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f453bf7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f4534106eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f453bf7d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4534107480 con 0x7f4534106a90 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f4539d19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f4534106eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f4539d19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f4534106eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36312/0 (socket says 172.21.15.31:36312) 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.860+0000 7f4539d19700 1 -- 172.21.15.31:0/2746919746 learned_addr learned my addr 172.21.15.31:0/2746919746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f4539d19700 1 -- 172.21.15.31:0/2746919746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4534107c90 con 0x7f4534106a90 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f4539d19700 1 --2- 172.21.15.31:0/2746919746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f4534106eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45300098a0 tx=0x7f4530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a18c63778c344c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f4538d17700 1 -- 172.21.15.31:0/2746919746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4530004160 con 0x7f4534106a90 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f4538d17700 1 -- 172.21.15.31:0/2746919746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4530004300 con 0x7f4534106a90 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f4538d17700 1 -- 172.21.15.31:0/2746919746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4530011a80 con 0x7f4534106a90 2024-04-25T00:21:01.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f453bf7d700 1 -- 172.21.15.31:0/2746919746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 msgr2=0x7f4534106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.861+0000 7f453bf7d700 1 --2- 172.21.15.31:0/2746919746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f4534106eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45300098a0 tx=0x7f4530009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 -- 172.21.15.31:0/2746919746 shutdown_connections 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 --2- 172.21.15.31:0/2746919746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f4534106eb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 -- 172.21.15.31:0/2746919746 >> 172.21.15.31:0/2746919746 conn(0x7f4534102050 msgr2=0x7f4534104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 -- 172.21.15.31:0/2746919746 shutdown_connections 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 -- 172.21.15.31:0/2746919746 wait complete. 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 Processor -- start 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.862+0000 7f453bf7d700 1 -- start start 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f453bf7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f453407b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f453bf7d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f453410b040 con 0x7f4534106a90 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f4539d19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f453407b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f4539d19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f453407b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36314/0 (socket says 172.21.15.31:36314) 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f4539d19700 1 -- 172.21.15.31:0/3121180382 learned_addr learned my addr 172.21.15.31:0/3121180382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f4539d19700 1 -- 172.21.15.31:0/3121180382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453419c620 con 0x7f4534106a90 2024-04-25T00:21:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f4539d19700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f453407b9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4530000c00 tx=0x7f4530011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45300039f0 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45341a0c30 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4530003b90 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f453001a740 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.863+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45341a0fd0 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.864+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f453001aa40 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.864+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4534063890 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.864+0000 7f452affd700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4520039050 0x7f452003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.864+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f453004d570 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.865+0000 7f4539518700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4520039050 0x7f452003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.865+0000 7f4539518700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4520039050 0x7f452003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4524006fd0 tx=0x7f4524006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:20:59.869+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4530004300 con 0x7f4534106a90 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.094+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4534103250 con 0x7f4520039050 2024-04-25T00:21:01.405 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.098+0000 7f452affd700 1 -- 172.21.15.31:0/3121180382 <== mgr.14118 v2:172.21.15.31:6800/3443312222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4534103250 con 0x7f4520039050 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.100+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4520039050 msgr2=0x7f452003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.100+0000 7f453bf7d700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4520039050 0x7f452003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4524006fd0 tx=0x7f4524006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.100+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 msgr2=0x7f453407b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.100+0000 7f453bf7d700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f453407b9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4530000c00 tx=0x7f4530011b00 comp rx=0 tx=0).stop 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.101+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 shutdown_connections 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.101+0000 7f453bf7d700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4520039050 0x7f452003b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.101+0000 7f453bf7d700 1 --2- 172.21.15.31:0/3121180382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4534106a90 0x7f453407b9f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.101+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 >> 172.21.15.31:0/3121180382 conn(0x7f4534102050 msgr2=0x7f4534102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.101+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 shutdown_connections 2024-04-25T00:21:01.406 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:00.101+0000 7f453bf7d700 1 -- 172.21.15.31:0/3121180382 wait complete. 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.863+0000 7fa8b5161700 1 Processor -- start 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.863+0000 7fa8b5161700 1 -- start start 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8b5161700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8b5161700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0107470 con 0x7fa8b0106a80 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36320/0 (socket says 172.21.15.31:36320) 2024-04-25T00:21:02.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8aed9d700 1 -- 172.21.15.31:0/1026540198 learned_addr learned my addr 172.21.15.31:0/1026540198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8aed9d700 1 -- 172.21.15.31:0/1026540198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b0107c80 con 0x7fa8b0106a80 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.864+0000 7fa8aed9d700 1 --2- 172.21.15.31:0/1026540198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0106ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00098a0 tx=0x7fa8a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a87341ebd7a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.865+0000 7fa8ae59c700 1 -- 172.21.15.31:0/1026540198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8a0004160 con 0x7fa8b0106a80 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.865+0000 7fa8ae59c700 1 -- 172.21.15.31:0/1026540198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8a0004300 con 0x7fa8b0106a80 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.865+0000 7fa8ae59c700 1 -- 172.21.15.31:0/1026540198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8a0011a50 con 0x7fa8b0106a80 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 -- 172.21.15.31:0/1026540198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 msgr2=0x7fa8b0106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 --2- 172.21.15.31:0/1026540198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0106ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00098a0 tx=0x7fa8a0009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 -- 172.21.15.31:0/1026540198 shutdown_connections 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 --2- 172.21.15.31:0/1026540198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0106ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 -- 172.21.15.31:0/1026540198 >> 172.21.15.31:0/1026540198 conn(0x7fa8b0102020 msgr2=0x7fa8b0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 -- 172.21.15.31:0/1026540198 shutdown_connections 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 -- 172.21.15.31:0/1026540198 wait complete. 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.866+0000 7fa8b5161700 1 Processor -- start 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8b5161700 1 -- start start 2024-04-25T00:21:02.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8b5161700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0191bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8b5161700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010b030 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0191bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0191bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36332/0 (socket says 172.21.15.31:36332) 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8aed9d700 1 -- 172.21.15.31:0/1025356217 learned_addr learned my addr 172.21.15.31:0/1025356217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8aed9d700 1 -- 172.21.15.31:0/1025356217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b0192130 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8aed9d700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0191bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0006ba0 tx=0x7fa8a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.867+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8a0003af0 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.868+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8a001a8e0 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.868+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b0190320 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.868+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8a0024900 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.868+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b0190860 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.868+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa8a0022020 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.869+0000 7fa8acd99700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fa89c039000 0x7fa89c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.869+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa8a004d5c0 con 0x7fa8b0106a80 2024-04-25T00:21:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.869+0000 7fa8a7fff700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fa89c039000 0x7fa89c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.869+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b0063890 con 0x7fa8b0106a80 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.869+0000 7fa8a7fff700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fa89c039000 0x7fa89c03b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa898006fd0 tx=0x7fa898006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:01.874+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa8a000b5a0 con 0x7fa8b0106a80 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.082+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa8b0195820 con 0x7fa8b0106a80 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.087+0000 7fa8acd99700 1 -- 172.21.15.31:0/1025356217 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fa8a0032030 con 0x7fa8b0106a80 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fa89c039000 msgr2=0x7fa89c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fa89c039000 0x7fa89c03b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa898006fd0 tx=0x7fa898006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 msgr2=0x7fa8b0191bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0191bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0006ba0 tx=0x7fa8a0011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 shutdown_connections 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fa89c039000 0x7fa89c03b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 --2- 172.21.15.31:0/1025356217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0106a80 0x7fa8b0191bf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 >> 172.21.15.31:0/1025356217 conn(0x7fa8b0102020 msgr2=0x7fa8b0102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:02.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 shutdown_connections 2024-04-25T00:21:02.679 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:02.090+0000 7fa8b5161700 1 -- 172.21.15.31:0/1025356217 wait complete. 2024-04-25T00:21:02.917 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:02 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:21:02.917 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:02 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1025356217' entity='client.admin' 2024-04-25T00:21:04.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:04 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3360758743' entity='client.admin' 2024-04-25T00:21:04.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.109+0000 7fcb7759e700 1 Processor -- start 2024-04-25T00:21:04.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.109+0000 7fcb7759e700 1 -- start start 2024-04-25T00:21:04.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:04.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7807c7c0 con 0x7fcb7807ddd0 2024-04-25T00:21:04.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:04.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57162/0 (socket says 172.21.15.31:57162) 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7659c700 1 -- 172.21.15.31:0/827115375 learned_addr learned my addr 172.21.15.31:0/827115375 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7659c700 1 -- 172.21.15.31:0/827115375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7807c940 con 0x7fcb7807ddd0 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.110+0000 7fcb7659c700 1 --2- 172.21.15.31:0/827115375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7807c280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb680098a0 tx=0x7fcb68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c85c0a3703f21bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7559a700 1 -- 172.21.15.31:0/827115375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb68004160 con 0x7fcb7807ddd0 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7559a700 1 -- 172.21.15.31:0/827115375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb68004300 con 0x7fcb7807ddd0 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7559a700 1 -- 172.21.15.31:0/827115375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb68011a50 con 0x7fcb7807ddd0 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 -- 172.21.15.31:0/827115375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 msgr2=0x7fcb7807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 --2- 172.21.15.31:0/827115375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7807c280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb680098a0 tx=0x7fcb68009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 -- 172.21.15.31:0/827115375 shutdown_connections 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 --2- 172.21.15.31:0/827115375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7807c280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 -- 172.21.15.31:0/827115375 >> 172.21.15.31:0/827115375 conn(0x7fcb78102ff0 msgr2=0x7fcb78105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 -- 172.21.15.31:0/827115375 shutdown_connections 2024-04-25T00:21:04.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.111+0000 7fcb7759e700 1 -- 172.21.15.31:0/827115375 wait complete. 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7759e700 1 Processor -- start 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7759e700 1 -- start start 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7759e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7810df30 con 0x7fcb7807ddd0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57178/0 (socket says 172.21.15.31:57178) 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.112+0000 7fcb7659c700 1 -- 172.21.15.31:0/3360758743 learned_addr learned my addr 172.21.15.31:0/3360758743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb7659c700 1 -- 172.21.15.31:0/3360758743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7819df90 con 0x7fcb7807ddd0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb7659c700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7819da50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb68006b20 tx=0x7fcb68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb6801a430 con 0x7fcb7807ddd0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6801aa80 con 0x7fcb7807ddd0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb68024a20 con 0x7fcb7807ddd0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7819e220 con 0x7fcb7807ddd0 2024-04-25T00:21:04.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.113+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb781a0f10 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.114+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcb68022020 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.114+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb78063890 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.114+0000 7fcb677fe700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fcb60039050 0x7fcb6003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.114+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb6804d800 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.114+0000 7fcb75d9b700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fcb60039050 0x7fcb6003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.115+0000 7fcb75d9b700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fcb60039050 0x7fcb6003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c006fd0 tx=0x7fcb6c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.119+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcb680042a0 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.360+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fcb781a1530 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.363+0000 7fcb677fe700 1 -- 172.21.15.31:0/3360758743 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fcb6802c3a0 con 0x7fcb7807ddd0 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fcb60039050 msgr2=0x7fcb6003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fcb60039050 0x7fcb6003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c006fd0 tx=0x7fcb6c006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 msgr2=0x7fcb7819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7819da50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb68006b20 tx=0x7fcb68011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 shutdown_connections 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7fcb60039050 0x7fcb6003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 --2- 172.21.15.31:0/3360758743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7807ddd0 0x7fcb7819da50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 >> 172.21.15.31:0/3360758743 conn(0x7fcb78102ff0 msgr2=0x7fcb78103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 shutdown_connections 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:03.365+0000 7fcb7759e700 1 -- 172.21.15.31:0/3360758743 wait complete. 2024-04-25T00:21:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-04-25T00:21:06.289 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-mon[29167]: from='mgr.14118 172.21.15.31:0/2096171759' entity='mgr.x' 2024-04-25T00:21:06.289 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1715123237' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T00:21:06.289 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: ignoring --setuser ceph since I am not root 2024-04-25T00:21:06.289 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: ignoring --setgroup ceph since I am not root 2024-04-25T00:21:06.289 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:06.221+0000 7feab573d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:21:06.542 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:06.434+0000 7feab573d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:06.542 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:06.540+0000 7feab573d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:21:06.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.932+0000 7f5ca846a700 1 Processor -- start 2024-04-25T00:21:06.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.932+0000 7f5ca846a700 1 -- start start 2024-04-25T00:21:06.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca846a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca010a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:06.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca846a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca007be50 con 0x7f5ca01085f0 2024-04-25T00:21:06.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca6206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca010a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca6206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca010a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57190/0 (socket says 172.21.15.31:57190) 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca6206700 1 -- 172.21.15.31:0/2081128600 learned_addr learned my addr 172.21.15.31:0/2081128600 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca6206700 1 -- 172.21.15.31:0/2081128600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca010af20 con 0x7f5ca01085f0 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.933+0000 7f5ca6206700 1 --2- 172.21.15.31:0/2081128600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca010a9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c940098a0 tx=0x7f5c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa9cc01187b2088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca5204700 1 -- 172.21.15.31:0/2081128600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c94004160 con 0x7f5ca01085f0 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca5204700 1 -- 172.21.15.31:0/2081128600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c94004300 con 0x7f5ca01085f0 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca5204700 1 -- 172.21.15.31:0/2081128600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c94011a50 con 0x7f5ca01085f0 2024-04-25T00:21:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 -- 172.21.15.31:0/2081128600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 msgr2=0x7f5ca010a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 --2- 172.21.15.31:0/2081128600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca010a9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c940098a0 tx=0x7f5c94009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 -- 172.21.15.31:0/2081128600 shutdown_connections 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 --2- 172.21.15.31:0/2081128600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca010a9e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 -- 172.21.15.31:0/2081128600 >> 172.21.15.31:0/2081128600 conn(0x7f5ca0102020 msgr2=0x7f5ca0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 -- 172.21.15.31:0/2081128600 shutdown_connections 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.934+0000 7f5ca846a700 1 -- 172.21.15.31:0/2081128600 wait complete. 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca846a700 1 Processor -- start 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca846a700 1 -- start start 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca846a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca846a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca010bf30 con 0x7f5ca01085f0 2024-04-25T00:21:06.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca6206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca6206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57196/0 (socket says 172.21.15.31:57196) 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.935+0000 7f5ca6206700 1 -- 172.21.15.31:0/1715123237 learned_addr learned my addr 172.21.15.31:0/1715123237 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5ca6206700 1 -- 172.21.15.31:0/1715123237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca019bf60 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5ca6206700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca019ba20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c94006b20 tx=0x7f5c94011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c94003b70 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c9401a8e0 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca019c1f0 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c940249b0 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.936+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca019eee0 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.937+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5c94022020 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.937+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca0063890 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.937+0000 7f5c937fe700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f5c8c0390a0 0x7f5c8c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.937+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c9404d7a0 con 0x7f5ca01085f0 2024-04-25T00:21:06.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.937+0000 7f5ca5a05700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f5c8c0390a0 0x7f5c8c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.938+0000 7f5ca5a05700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f5c8c0390a0 0x7f5c8c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006fd0 tx=0x7f5c9c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:04.942+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5c9402c3a0 con 0x7f5ca01085f0 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:05.215+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5ca019f490 con 0x7f5ca01085f0 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.035+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5c94017ca0 con 0x7f5ca01085f0 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.037+0000 7f5c937fe700 1 -- 172.21.15.31:0/1715123237 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5c9401a430 con 0x7f5ca01085f0 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f5c8c0390a0 msgr2=0x7f5c8c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f5c8c0390a0 0x7f5c8c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006fd0 tx=0x7f5c9c006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 msgr2=0x7f5ca019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca019ba20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c94006b20 tx=0x7f5c94011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 shutdown_connections 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f5c8c0390a0 0x7f5c8c03b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 --2- 172.21.15.31:0/1715123237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ca01085f0 0x7f5ca019ba20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 >> 172.21.15.31:0/1715123237 conn(0x7f5ca0102020 msgr2=0x7f5ca0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 shutdown_connections 2024-04-25T00:21:06.654 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:06.039+0000 7f5ca846a700 1 -- 172.21.15.31:0/1715123237 wait complete. 2024-04-25T00:21:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:07 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1715123237' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T00:21:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:07 smithi031 ceph-mon[29167]: mgrmap e9: x(active, since 20s) 2024-04-25T00:21:07.272 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:06.873+0000 7feab573d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:21:07.272 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:06.965+0000 7feab573d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:21:08.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.088+0000 7f39dc35d700 1 Processor -- start 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.088+0000 7f39dc35d700 1 -- start start 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.088+0000 7f39dc35d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4106230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.088+0000 7f39dc35d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4106800 con 0x7f39d4105e10 2024-04-25T00:21:08.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.089+0000 7f39da0f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.089+0000 7f39da0f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4106230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57218/0 (socket says 172.21.15.31:57218) 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.089+0000 7f39da0f9700 1 -- 172.21.15.31:0/2545519475 learned_addr learned my addr 172.21.15.31:0/2545519475 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.089+0000 7f39da0f9700 1 -- 172.21.15.31:0/2545519475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4106980 con 0x7f39d4105e10 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.089+0000 7f39da0f9700 1 --2- 172.21.15.31:0/2545519475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4106230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098a0 tx=0x7f39c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc504f1027af6cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.089+0000 7f39d90f7700 1 -- 172.21.15.31:0/2545519475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39c4004160 con 0x7f39d4105e10 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39d90f7700 1 -- 172.21.15.31:0/2545519475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c4004300 con 0x7f39d4105e10 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39d90f7700 1 -- 172.21.15.31:0/2545519475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39c4011a50 con 0x7f39d4105e10 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 -- 172.21.15.31:0/2545519475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 msgr2=0x7f39d4106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 --2- 172.21.15.31:0/2545519475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4106230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098a0 tx=0x7f39c4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 -- 172.21.15.31:0/2545519475 shutdown_connections 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 --2- 172.21.15.31:0/2545519475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4106230 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 -- 172.21.15.31:0/2545519475 >> 172.21.15.31:0/2545519475 conn(0x7f39d4101790 msgr2=0x7f39d4103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 -- 172.21.15.31:0/2545519475 shutdown_connections 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.090+0000 7f39dc35d700 1 -- 172.21.15.31:0/2545519475 wait complete. 2024-04-25T00:21:08.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39dc35d700 1 Processor -- start 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39dc35d700 1 -- start start 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39dc35d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4108710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39dc35d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410c0c0 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39da0f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39da0f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57220/0 (socket says 172.21.15.31:57220) 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39da0f9700 1 -- 172.21.15.31:0/3232078010 learned_addr learned my addr 172.21.15.31:0/3232078010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.091+0000 7f39da0f9700 1 -- 172.21.15.31:0/3232078010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4108c50 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.092+0000 7f39da0f9700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4108710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39c4006b20 tx=0x7f39c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.092+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39c40039e0 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.092+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c401a8e0 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.092+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d4106e40 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.092+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39c40247e0 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.092+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d4107380 con 0x7f39d4105e10 2024-04-25T00:21:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.093+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f39c4022020 con 0x7f39d4105e10 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.093+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4063890 con 0x7f39d4105e10 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.093+0000 7f39cb7fe700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 0x7f39c003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.093+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f39c404d4a0 con 0x7f39d4105e10 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.093+0000 7f39d98f8700 1 -- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 msgr2=0x7f39c003b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.093+0000 7f39d98f8700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 0x7f39c003b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.098+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39c400b3e0 con 0x7f39d4105e10 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.294+0000 7f39d98f8700 1 -- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 msgr2=0x7f39c003b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.294+0000 7f39d98f8700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 0x7f39c003b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.386+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f39d41a1ad0 con 0x7f39d4105e10 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.387+0000 7f39cb7fe700 1 -- 172.21.15.31:0/3232078010 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f39c4020c10 con 0x7f39d4105e10 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 msgr2=0x7f39c003b5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 0x7f39c003b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 msgr2=0x7f39d4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4108710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39c4006b20 tx=0x7f39c4011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 shutdown_connections 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f39c00390f0 0x7f39c003b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 --2- 172.21.15.31:0/3232078010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d4105e10 0x7f39d4108710 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39c4006b20 tx=0x7f39c4011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.391+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 >> 172.21.15.31:0/3232078010 conn(0x7f39d4101790 msgr2=0x7f39d4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.392+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 shutdown_connections 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:07.392+0000 7f39dc35d700 1 -- 172.21.15.31:0/3232078010 wait complete. 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-25T00:21:08.018 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-04-25T00:21:08.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:08 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3232078010' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:21:08.272 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:08.243+0000 7feab573d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:21:08.606 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:08.426+0000 7feab573d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:08.606 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:08.604+0000 7feab573d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:21:09.212 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:08.904+0000 7feab573d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:21:09.212 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:09.001+0000 7feab573d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:21:09.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:09.211+0000 7feab573d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:21:09.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:09.411+0000 7feab573d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:10.272 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:10 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:10.039+0000 7feab573d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:21:10.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:10 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:10.479+0000 7feab573d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:21:10.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:10 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:10.579+0000 7feab573d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:21:11.272 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:10 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:10.792+0000 7feab573d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:21:11.692 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:11.338+0000 7feab573d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:21:11.693 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:11.438+0000 7feab573d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:21:12.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:11.692+0000 7feab573d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:21:12.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:11.831+0000 7feab573d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:21:12.022 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:11.930+0000 7feab573d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:21:12.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:12 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:12.051+0000 7feab573d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:12.522 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:12 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:12.147+0000 7feab573d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:21:13.270 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:12 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:12.992+0000 7feab573d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:21:13.270 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:21:13.216+0000 7feab573d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:21:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: Active manager daemon x restarted 2024-04-25T00:21:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: Activating manager daemon x 2024-04-25T00:21:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T00:21:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: mgrmap e10: x(active, starting, since 0.00342257s) 2024-04-25T00:21:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:21:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:21:13.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:21:13.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:21:13.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:21:13.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: Manager daemon x is now available 2024-04-25T00:21:13.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:13.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:14.450 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:14.450 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:14 smithi031 ceph-mon[29167]: [25/Apr/2024:00:21:14] ENGINE Bus STARTING 2024-04-25T00:21:14.450 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:14 smithi031 ceph-mon[29167]: mgrmap e11: x(active, since 1.00581s) 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bba9c0700 1 Processor -- start 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bba9c0700 1 -- start start 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bba9c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb410aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bba9c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb407bdf0 con 0x7f4bb4108620 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb410aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb410aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57228/0 (socket says 172.21.15.31:57228) 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bb3fff700 1 -- 172.21.15.31:0/1931841095 learned_addr learned my addr 172.21.15.31:0/1931841095 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bb3fff700 1 -- 172.21.15.31:0/1931841095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb410af50 con 0x7f4bb4108620 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.426+0000 7f4bb3fff700 1 --2- 172.21.15.31:0/1931841095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb410aa10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40098a0 tx=0x7f4ba4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17e3606e45ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bb2ffd700 1 -- 172.21.15.31:0/1931841095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4004160 con 0x7f4bb4108620 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bb2ffd700 1 -- 172.21.15.31:0/1931841095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba4004300 con 0x7f4bb4108620 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bb2ffd700 1 -- 172.21.15.31:0/1931841095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4011a50 con 0x7f4bb4108620 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 -- 172.21.15.31:0/1931841095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 msgr2=0x7f4bb410aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/1931841095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb410aa10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40098a0 tx=0x7f4ba4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 -- 172.21.15.31:0/1931841095 shutdown_connections 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/1931841095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb410aa10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 -- 172.21.15.31:0/1931841095 >> 172.21.15.31:0/1931841095 conn(0x7f4bb4101ff0 msgr2=0x7f4bb4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 -- 172.21.15.31:0/1931841095 shutdown_connections 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.427+0000 7f4bba9c0700 1 -- 172.21.15.31:0/1931841095 wait complete. 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.428+0000 7f4bba9c0700 1 Processor -- start 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.428+0000 7f4bba9c0700 1 -- start start 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.428+0000 7f4bba9c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.428+0000 7f4bba9c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb410bf00 con 0x7f4bb4108620 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57236/0 (socket says 172.21.15.31:57236) 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb3fff700 1 -- 172.21.15.31:0/3137532828 learned_addr learned my addr 172.21.15.31:0/3137532828 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb3fff700 1 -- 172.21.15.31:0/3137532828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb419bf40 con 0x7f4bb4108620 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb3fff700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb419ba00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4011a20 tx=0x7f4ba4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4003b70 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba401a8e0 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb419c1d0 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba4024990 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.429+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb419eec0 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.430+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f4ba4022020 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.431+0000 7f4bb17fa700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.431+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 --> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b9c03bc20 con 0x7f4b9c039050 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.431+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ba404d3f0 con 0x7f4bb4108620 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.431+0000 7f4bb37fe700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 msgr2=0x7f4b9c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:15.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.431+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.632+0000 7f4bb37fe700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 msgr2=0x7f4b9c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:08.632+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:09.033+0000 7f4bb37fe700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 msgr2=0x7f4b9c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:09.033+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:09.834+0000 7f4bb37fe700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 msgr2=0x7f4b9c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:09.834+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:11.436+0000 7f4bb37fe700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 msgr2=0x7f4b9c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3443312222 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:11.436+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:13.220+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f4ba401aa90 con 0x7f4bb4108620 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:13.220+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 msgr2=0x7f4b9c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:13.220+0000 7f4bb17fa700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.221+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f4ba404f230 con 0x7f4bb4108620 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.221+0000 7f4bb17fa700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b9c03cb10 0x7f4b9c03ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.221+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b9c03bc20 con 0x7f4b9c03cb10 2024-04-25T00:21:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.222+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b9c03cb10 0x7f4b9c03ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.222+0000 7f4bb37fe700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b9c03cb10 0x7f4b9c03ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8003a10 tx=0x7f4ba80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.223+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4b9c03bc20 con 0x7f4b9c03cb10 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.230+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4bb4063890 con 0x7f4b9c03cb10 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.230+0000 7f4bb17fa700 1 -- 172.21.15.31:0/3137532828 <== mgr.14150 v2:172.21.15.31:6800/2153348461 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4bb4063890 con 0x7f4b9c03cb10 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b9c03cb10 msgr2=0x7f4b9c03ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b9c03cb10 0x7f4b9c03ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8003a10 tx=0x7f4ba80092b0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 msgr2=0x7f4bb419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb419ba00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4011a20 tx=0x7f4ba4011b00 comp rx=0 tx=0).stop 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 shutdown_connections 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b9c03cb10 0x7f4b9c03ef00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb4108620 0x7f4bb419ba00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 --2- 172.21.15.31:0/3137532828 >> [v2:172.21.15.31:6800/3443312222,v1:172.21.15.31:6801/3443312222] conn(0x7f4b9c039050 0x7f4b9c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:15.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 >> 172.21.15.31:0/3137532828 conn(0x7f4bb4101ff0 msgr2=0x7f4bb4102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:15.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 shutdown_connections 2024-04-25T00:21:15.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:14.231+0000 7f4bba9c0700 1 -- 172.21.15.31:0/3137532828 wait complete. 2024-04-25T00:21:15.197 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-04-25T00:21:15.197 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-04-25T00:21:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:15 smithi031 ceph-mon[29167]: [25/Apr/2024:00:21:14] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T00:21:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:15 smithi031 ceph-mon[29167]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:21:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:15 smithi031 ceph-mon[29167]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:21:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:15 smithi031 ceph-mon[29167]: [25/Apr/2024:00:21:14] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T00:21:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:15 smithi031 ceph-mon[29167]: [25/Apr/2024:00:21:14] ENGINE Bus STARTED 2024-04-25T00:21:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa924fbd700 1 Processor -- start 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa924fbd700 1 -- start start 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa924fbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa92010a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa924fbd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa92007be50 con 0x7fa9201085e0 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa91e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa92010a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa91e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa92010a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46852/0 (socket says 172.21.15.31:46852) 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.613+0000 7fa91e59c700 1 -- 172.21.15.31:0/133961752 learned_addr learned my addr 172.21.15.31:0/133961752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.614+0000 7fa91e59c700 1 -- 172.21.15.31:0/133961752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92010af10 con 0x7fa9201085e0 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.614+0000 7fa91e59c700 1 --2- 172.21.15.31:0/133961752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa92010a9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098a0 tx=0x7fa910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd81954316d97f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.614+0000 7fa91d59a700 1 -- 172.21.15.31:0/133961752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910004160 con 0x7fa9201085e0 2024-04-25T00:21:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.614+0000 7fa91d59a700 1 -- 172.21.15.31:0/133961752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910004300 con 0x7fa9201085e0 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.614+0000 7fa91d59a700 1 -- 172.21.15.31:0/133961752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910011a50 con 0x7fa9201085e0 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 -- 172.21.15.31:0/133961752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 msgr2=0x7fa92010a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 --2- 172.21.15.31:0/133961752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa92010a9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098a0 tx=0x7fa910009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 -- 172.21.15.31:0/133961752 shutdown_connections 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 --2- 172.21.15.31:0/133961752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa92010a9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 -- 172.21.15.31:0/133961752 >> 172.21.15.31:0/133961752 conn(0x7fa920102010 msgr2=0x7fa920104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 -- 172.21.15.31:0/133961752 shutdown_connections 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.615+0000 7fa924fbd700 1 -- 172.21.15.31:0/133961752 wait complete. 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa924fbd700 1 Processor -- start 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa924fbd700 1 -- start start 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa924fbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa920107e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa924fbd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010bf20 con 0x7fa9201085e0 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa91e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa920107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa91e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa920107e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46860/0 (socket says 172.21.15.31:46860) 2024-04-25T00:21:16.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa91e59c700 1 -- 172.21.15.31:0/532874892 learned_addr learned my addr 172.21.15.31:0/532874892 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:16.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.616+0000 7fa91e59c700 1 -- 172.21.15.31:0/532874892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9201083a0 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.617+0000 7fa91e59c700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa920107e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa910011a20 tx=0x7fa910011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.617+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910003b70 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.617+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa91001a8e0 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.617+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa920106590 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.617+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910024990 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.617+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa920106ad0 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.618+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa910022020 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.618+0000 7fa90f7fe700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa908038f40 0x7fa90803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.618+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa91004d6f0 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.618+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa920063890 con 0x7fa9201085e0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.618+0000 7fa91dd9b700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa908038f40 0x7fa90803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.619+0000 7fa91dd9b700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa908038f40 0x7fa90803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.623+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa91000b3e0 con 0x7fa9201085e0 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.830+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa91002dc90 con 0x7fa9201085e0 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.838+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa9201073b0 con 0x7fa908038f40 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.988+0000 7fa90f7fe700 1 -- 172.21.15.31:0/532874892 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa9201073b0 con 0x7fa908038f40 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa908038f40 msgr2=0x7fa90803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa908038f40 0x7fa90803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 msgr2=0x7fa920107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa920107e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa910011a20 tx=0x7fa910011b00 comp rx=0 tx=0).stop 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 shutdown_connections 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa908038f40 0x7fa90803b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 --2- 172.21.15.31:0/532874892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9201085e0 0x7fa920107e60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 >> 172.21.15.31:0/532874892 conn(0x7fa920102010 msgr2=0x7fa920102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 shutdown_connections 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:15.990+0000 7fa924fbd700 1 -- 172.21.15.31:0/532874892 wait complete. 2024-04-25T00:21:16.591 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-04-25T00:21:17.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:16 smithi031 ceph-mon[29167]: mgrmap e12: x(active, since 2s) 2024-04-25T00:21:17.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:16 smithi031 ceph-mon[29167]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:17.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:17.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:17.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$obcdv1vsAvjxsrwB257GUOCD5K0mn9WuW0ZAR5eje8YQaD.NfVkTS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714004477, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66aef8d700 1 Processor -- start 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66aef8d700 1 -- start start 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66aef8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a8106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66aef8d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8106830 con 0x7f66a8105e40 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66acd29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a8106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66acd29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a8106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46870/0 (socket says 172.21.15.31:46870) 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66acd29700 1 -- 172.21.15.31:0/1324855627 learned_addr learned my addr 172.21.15.31:0/1324855627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.058+0000 7f66acd29700 1 -- 172.21.15.31:0/1324855627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a8108dd0 con 0x7f66a8105e40 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.059+0000 7f66acd29700 1 --2- 172.21.15.31:0/1324855627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a8106260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f669c0098a0 tx=0x7f669c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5a0fefe8342ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.059+0000 7f66a77fe700 1 -- 172.21.15.31:0/1324855627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669c004160 con 0x7f66a8105e40 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.059+0000 7f66a77fe700 1 -- 172.21.15.31:0/1324855627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669c004300 con 0x7f66a8105e40 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.059+0000 7f66a77fe700 1 -- 172.21.15.31:0/1324855627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669c011a50 con 0x7f66a8105e40 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 -- 172.21.15.31:0/1324855627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 msgr2=0x7f66a8106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 --2- 172.21.15.31:0/1324855627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a8106260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f669c0098a0 tx=0x7f669c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 -- 172.21.15.31:0/1324855627 shutdown_connections 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 --2- 172.21.15.31:0/1324855627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a8106260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 -- 172.21.15.31:0/1324855627 >> 172.21.15.31:0/1324855627 conn(0x7f66a8101790 msgr2=0x7f66a8103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 -- 172.21.15.31:0/1324855627 shutdown_connections 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 -- 172.21.15.31:0/1324855627 wait complete. 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 Processor -- start 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.060+0000 7f66aef8d700 1 -- start start 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66aef8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66aef8d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810c180 con 0x7f66a8105e40 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66acd29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66acd29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46872/0 (socket says 172.21.15.31:46872) 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66acd29700 1 -- 172.21.15.31:0/2315778885 learned_addr learned my addr 172.21.15.31:0/2315778885 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66acd29700 1 -- 172.21.15.31:0/2315778885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a819bfd0 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66acd29700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a819ba90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f669c006b20 tx=0x7f669c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669c0039e0 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669c01a8e0 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a819c260 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.061+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669c0247e0 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.062+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a819ef50 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.062+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f669c022020 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.063+0000 7f66a5ffb700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66980390a0 0x7f669803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.063+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f669c04d590 con 0x7f66a8105e40 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.063+0000 7f66a7fff700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66980390a0 0x7f669803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.063+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a8063890 con 0x7f66a8105e40 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.063+0000 7f66a7fff700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66980390a0 0x7f669803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6694006fd0 tx=0x7f6694006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.072+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f669c00b3e0 con 0x7f66a8105e40 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.314+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f66a819f120 con 0x7f66980390a0 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.561+0000 7f66a5ffb700 1 -- 172.21.15.31:0/2315778885 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f66a819f120 con 0x7f66980390a0 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66980390a0 msgr2=0x7f669803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66980390a0 0x7f669803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6694006fd0 tx=0x7f6694006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 msgr2=0x7f66a819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a819ba90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f669c006b20 tx=0x7f669c011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 shutdown_connections 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66980390a0 0x7f669803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 --2- 172.21.15.31:0/2315778885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a8105e40 0x7f66a819ba90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 >> 172.21.15.31:0/2315778885 conn(0x7f66a8101790 msgr2=0x7f66a8102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 shutdown_connections 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:17.563+0000 7f66aef8d700 1 -- 172.21.15.31:0/2315778885 wait complete. 2024-04-25T00:21:18.319 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-04-25T00:21:18.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:18 smithi031 ceph-mon[29167]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:18.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:19.866 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:19 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3739640889' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.763+0000 7f3e2b1cb700 1 Processor -- start 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.763+0000 7f3e2b1cb700 1 -- start start 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.763+0000 7f3e2b1cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e2410aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.763+0000 7f3e2b1cb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2407be50 con 0x7f3e24108610 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e28f67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e2410aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e28f67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e2410aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46884/0 (socket says 172.21.15.31:46884) 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e28f67700 1 -- 172.21.15.31:0/1792924291 learned_addr learned my addr 172.21.15.31:0/1792924291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e28f67700 1 -- 172.21.15.31:0/1792924291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2410af40 con 0x7f3e24108610 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e28f67700 1 --2- 172.21.15.31:0/1792924291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e2410aa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e140098a0 tx=0x7f3e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1963dfd172a1baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e237fe700 1 -- 172.21.15.31:0/1792924291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14004160 con 0x7f3e24108610 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e237fe700 1 -- 172.21.15.31:0/1792924291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e14004300 con 0x7f3e24108610 2024-04-25T00:21:19.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.764+0000 7f3e237fe700 1 -- 172.21.15.31:0/1792924291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14011a50 con 0x7f3e24108610 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/1792924291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 msgr2=0x7f3e2410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 --2- 172.21.15.31:0/1792924291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e2410aa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e140098a0 tx=0x7f3e14009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/1792924291 shutdown_connections 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 --2- 172.21.15.31:0/1792924291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e2410aa00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/1792924291 >> 172.21.15.31:0/1792924291 conn(0x7f3e24102060 msgr2=0x7f3e24104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/1792924291 shutdown_connections 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/1792924291 wait complete. 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.765+0000 7f3e2b1cb700 1 Processor -- start 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e2b1cb700 1 -- start start 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e2b1cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e24107e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e2b1cb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410bf50 con 0x7f3e24108610 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e28f67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e24107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e28f67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e24107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46892/0 (socket says 172.21.15.31:46892) 2024-04-25T00:21:19.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e28f67700 1 -- 172.21.15.31:0/3739640889 learned_addr learned my addr 172.21.15.31:0/3739640889 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e28f67700 1 -- 172.21.15.31:0/3739640889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e241083d0 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e28f67700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e24107e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006b20 tx=0x7f3e14011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14003b70 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e1401a8e0 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e241065c0 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.766+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e140249b0 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.767+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e24106b00 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.767+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3e14022020 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.767+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e24063890 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.768+0000 7f3e21ffb700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.768+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e1404d910 con 0x7f3e24108610 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.768+0000 7f3e23fff700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.768+0000 7f3e23fff700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e0c039100 0x7f3e0c03b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e18006fd0 tx=0x7f3e18006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.773+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e1402c3a0 con 0x7f3e24108610 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.980+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3e24106e40 con 0x7f3e24108610 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.980+0000 7f3e21ffb700 1 -- 172.21.15.31:0/3739640889 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3e24106e40 con 0x7f3e24108610 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e0c039100 0x7f3e0c03b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e18006fd0 tx=0x7f3e18006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 msgr2=0x7f3e24107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e24107e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006b20 tx=0x7f3e14011ad0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 shutdown_connections 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 --2- 172.21.15.31:0/3739640889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e24108610 0x7f3e24107e90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 >> 172.21.15.31:0/3739640889 conn(0x7f3e24102060 msgr2=0x7f3e24102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 shutdown_connections 2024-04-25T00:21:19.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:18.983+0000 7f3e2b1cb700 1 -- 172.21.15.31:0/3739640889 wait complete. 2024-04-25T00:21:19.885 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:21:19.885 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T00:21:19.893 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:21:19.893 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T00:21:19.893 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T00:21:19.893 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T00:21:19.894 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-04-25T00:21:19.894 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:19.895 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031:8443/ 2024-04-25T00:21:19.895 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-04-25T00:21:19.895 INFO:teuthology.orchestra.run.smithi031.stdout: Password: umb83r5zqn 2024-04-25T00:21:19.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:19.895 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config directory 2024-04-25T00:21:19.895 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2024-04-25T00:21:20.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:20 smithi031 ceph-mon[29167]: mgrmap e13: x(active, since 6s) 2024-04-25T00:21:21.106 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.319+0000 7f60dbf54700 1 Processor -- start 2024-04-25T00:21:21.106 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.319+0000 7f60dbf54700 1 -- start start 2024-04-25T00:21:21.106 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60dbf54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d410aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.106 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60dbf54700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60d407be30 con 0x7f60d4108610 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d9cf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d410aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d9cf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d410aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46904/0 (socket says 172.21.15.31:46904) 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d9cf0700 1 -- 172.21.15.31:0/2119014761 learned_addr learned my addr 172.21.15.31:0/2119014761 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d9cf0700 1 -- 172.21.15.31:0/2119014761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d410af40 con 0x7f60d4108610 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d9cf0700 1 --2- 172.21.15.31:0/2119014761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d410aa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60d00098a0 tx=0x7f60d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4bd88d51bfd1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d8cee700 1 -- 172.21.15.31:0/2119014761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60d0004160 con 0x7f60d4108610 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d8cee700 1 -- 172.21.15.31:0/2119014761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60d0004300 con 0x7f60d4108610 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.320+0000 7f60d8cee700 1 -- 172.21.15.31:0/2119014761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60d0011a50 con 0x7f60d4108610 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 -- 172.21.15.31:0/2119014761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 msgr2=0x7f60d410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 --2- 172.21.15.31:0/2119014761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d410aa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60d00098a0 tx=0x7f60d0009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 -- 172.21.15.31:0/2119014761 shutdown_connections 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 --2- 172.21.15.31:0/2119014761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d410aa00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 -- 172.21.15.31:0/2119014761 >> 172.21.15.31:0/2119014761 conn(0x7f60d4102060 msgr2=0x7f60d4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 -- 172.21.15.31:0/2119014761 shutdown_connections 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.321+0000 7f60dbf54700 1 -- 172.21.15.31:0/2119014761 wait complete. 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60dbf54700 1 Processor -- start 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60dbf54700 1 -- start start 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60dbf54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60dbf54700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60d410bf40 con 0x7f60d4108610 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60d9cf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60d9cf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46916/0 (socket says 172.21.15.31:46916) 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60d9cf0700 1 -- 172.21.15.31:0/3624532343 learned_addr learned my addr 172.21.15.31:0/3624532343 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.322+0000 7f60d9cf0700 1 -- 172.21.15.31:0/3624532343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d419bf10 con 0x7f60d4108610 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.323+0000 7f60d9cf0700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d419b9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60d0011a20 tx=0x7f60d0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.323+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60d0003b70 con 0x7f60d4108610 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.323+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60d001a8e0 con 0x7f60d4108610 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.323+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d419c200 con 0x7f60d4108610 2024-04-25T00:21:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.323+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60d0024990 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.323+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d419ee90 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.324+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f60d0022020 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.324+0000 7f60caffd700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f60c0039150 0x7f60c003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.324+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d4063890 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.324+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f60d004dae0 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.324+0000 7f60d94ef700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f60c0039150 0x7f60c003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.324+0000 7f60d94ef700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f60c0039150 0x7f60c003b610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.329+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f60d000b3e0 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.535+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f60d419f180 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.535+0000 7f60caffd700 1 -- 172.21.15.31:0/3624532343 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f60d002dc90 con 0x7f60d4108610 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f60c0039150 msgr2=0x7f60c003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f60c0039150 0x7f60c003b610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 msgr2=0x7f60d419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.109 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d419b9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60d0011a20 tx=0x7f60d0011b00 comp rx=0 tx=0).stop 2024-04-25T00:21:21.110 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 shutdown_connections 2024-04-25T00:21:21.110 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f60c0039150 0x7f60c003b610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.110 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 --2- 172.21.15.31:0/3624532343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d4108610 0x7f60d419b9d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.110 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 >> 172.21.15.31:0/3624532343 conn(0x7f60d4102060 msgr2=0x7f60d4102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.110 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 shutdown_connections 2024-04-25T00:21:21.110 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:20.537+0000 7f60dbf54700 1 -- 172.21.15.31:0/3624532343 wait complete. 2024-04-25T00:21:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f746276f700 1 Processor -- start 2024-04-25T00:21:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f746276f700 1 -- start start 2024-04-25T00:21:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f746276f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c10aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f746276f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f745c07bdf0 con 0x7f745c108660 2024-04-25T00:21:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f745bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c10aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f745bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c10aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46920/0 (socket says 172.21.15.31:46920) 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.557+0000 7f745bfff700 1 -- 172.21.15.31:0/1833614445 learned_addr learned my addr 172.21.15.31:0/1833614445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f745bfff700 1 -- 172.21.15.31:0/1833614445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c10af90 con 0x7f745c108660 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f745bfff700 1 --2- 172.21.15.31:0/1833614445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c10aa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f744c0098a0 tx=0x7f744c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaa99d6540ca102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f745affd700 1 -- 172.21.15.31:0/1833614445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f744c004160 con 0x7f745c108660 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f745affd700 1 -- 172.21.15.31:0/1833614445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f744c004300 con 0x7f745c108660 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f745affd700 1 -- 172.21.15.31:0/1833614445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f744c011a50 con 0x7f745c108660 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f746276f700 1 -- 172.21.15.31:0/1833614445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 msgr2=0x7f745c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f746276f700 1 --2- 172.21.15.31:0/1833614445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c10aa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f744c0098a0 tx=0x7f744c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f746276f700 1 -- 172.21.15.31:0/1833614445 shutdown_connections 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.558+0000 7f746276f700 1 --2- 172.21.15.31:0/1833614445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c10aa50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.559+0000 7f746276f700 1 -- 172.21.15.31:0/1833614445 >> 172.21.15.31:0/1833614445 conn(0x7f745c102050 msgr2=0x7f745c104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.559+0000 7f746276f700 1 -- 172.21.15.31:0/1833614445 shutdown_connections 2024-04-25T00:21:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.559+0000 7f746276f700 1 -- 172.21.15.31:0/1833614445 wait complete. 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.559+0000 7f746276f700 1 Processor -- start 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.559+0000 7f746276f700 1 -- start start 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f746276f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c107ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f746276f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f745c10bf40 con 0x7f745c108660 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f745bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c107ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f745bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c107ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46936/0 (socket says 172.21.15.31:46936) 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f745bfff700 1 -- 172.21.15.31:0/1523057968 learned_addr learned my addr 172.21.15.31:0/1523057968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f745bfff700 1 -- 172.21.15.31:0/1523057968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c108420 con 0x7f745c108660 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f745bfff700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c107ee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f744c011a20 tx=0x7f744c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f744c003b70 con 0x7f745c108660 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f744c01a8e0 con 0x7f745c108660 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f744c024990 con 0x7f745c108660 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c106610 con 0x7f745c108660 2024-04-25T00:21:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.560+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c106b50 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.561+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f744c022020 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.561+0000 7f74597fa700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440390f0 0x7f744403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.561+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c063890 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.561+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f744c04d9d0 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.562+0000 7f745b7fe700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440390f0 0x7f744403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.562+0000 7f745b7fe700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440390f0 0x7f744403b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7450006fd0 tx=0x7f7450006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.566+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f744c00f2e0 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.893+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f745c106dd0 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.895+0000 7f74597fa700 1 -- 172.21.15.31:0/1523057968 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f744c02d3f0 con 0x7f745c108660 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440390f0 msgr2=0x7f744403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440390f0 0x7f744403b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7450006fd0 tx=0x7f7450006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:22.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 msgr2=0x7f745c107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c107ee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f744c011a20 tx=0x7f744c011b00 comp rx=0 tx=0).stop 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 shutdown_connections 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440390f0 0x7f744403b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 --2- 172.21.15.31:0/1523057968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c108660 0x7f745c107ee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 >> 172.21.15.31:0/1523057968 conn(0x7f745c102050 msgr2=0x7f745c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 shutdown_connections 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T00:21:21.898+0000 7f746276f700 1 -- 172.21.15.31:0/1523057968 wait complete. 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T00:21:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:22.568 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-04-25T00:21:22.604 INFO:tasks.cephadm:Fetching config... 2024-04-25T00:21:22.604 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:21:22.604 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T00:21:22.620 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T00:21:22.620 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:21:22.620 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T00:21:22.677 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T00:21:22.677 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:21:22.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T00:21:22.742 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T00:21:22.743 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:21:22.743 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T00:21:22.799 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T00:21:22.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:21:22.872 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:21:22.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:21:22.925 INFO:teuthology.orchestra.run.smithi040.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:21:22.937 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:21:22.976 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDppoi2JFNQ7UdsRG7CLD8W5/mCTmhPFmrPdRtkTlqarKQqRmvr1QsvXJZvZ7c0BdYTx1ZNvsBSo8w7id1N5+dWnNZHXsJ+Jahkr3bAV3xE6FlRe3h515fJ5FAu7QMDKSWL9bV2TZ4uVJsK8MlFvXch2IMsszcXIV+c+GwLyMLkngb3rIwJnZzR6DGBNwL53EGtgLstvcbuNTnl7q0z4xsEu1Up1Oyfb77D60cvTDviT00rF2nwFsRl+3Vob7UEkh8WWM8QK1tYFPThXX+NbT1X+MU0mft4FpqeE/R2NNUiXVUBziePlFc/Hm5IJg4qZZAxOSXgxr5j7eztPv7hphsnXVegvhhXVcVkKkHL40oMhsscyy1WWNfnVRBkUQF4pNIH6BhmyV7RdIqfmVpZjS4me+Rv+dX+yKdmq5BXUIkoeCLwoT/YBSJqsZVBYTQh7WmyaAqnQ6ROe67uPiFcOSI/xMQ69jSGcW3NWdJ0VbY1YfH7f9/1v5aiSj1sAjTfT20= ceph-6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:21:22.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T00:21:23.246 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:22 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1523057968' entity='client.admin' 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 -- 172.21.15.31:0/3243079241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 msgr2=0x7f42781037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 --2- 172.21.15.31:0/3243079241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f42781037c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4268009910 tx=0x7f4268009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 -- 172.21.15.31:0/3243079241 shutdown_connections 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 --2- 172.21.15.31:0/3243079241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f42781037c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 -- 172.21.15.31:0/3243079241 >> 172.21.15.31:0/3243079241 conn(0x7f42780fd280 msgr2=0x7f42780ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 -- 172.21.15.31:0/3243079241 shutdown_connections 2024-04-25T00:21:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.479+0000 7f427c8b8700 1 -- 172.21.15.31:0/3243079241 wait complete. 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427c8b8700 1 Processor -- start 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427c8b8700 1 -- start start 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427c8b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f4278075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427c8b8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4278107d10 con 0x7f42781033a0 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f4278075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f4278075ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40852/0 (socket says 172.21.15.31:40852) 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427659c700 1 -- 172.21.15.31:0/571653447 learned_addr learned my addr 172.21.15.31:0/571653447 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.480+0000 7f427659c700 1 -- 172.21.15.31:0/571653447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4278076430 con 0x7f42781033a0 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.481+0000 7f427659c700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f4278075ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4268005800 tx=0x7f4268009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.481+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4268027070 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.481+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f426800be10 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.481+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4278074620 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.481+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f426800f460 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.481+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4278074b60 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.482+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f426802a070 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.482+0000 7f42677fe700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4260039150 0x7f426003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.482+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4278108490 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.482+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f426801e030 con 0x7f42781033a0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.482+0000 7f4275d9b700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4260039150 0x7f426003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.482+0000 7f4275d9b700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4260039150 0x7f426003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f426c006fd0 tx=0x7f426c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.487+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4268013070 con 0x7f42781033a0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.697+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4278190680 con 0x7f42781033a0 2024-04-25T00:21:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.699+0000 7f42677fe700 1 -- 172.21.15.31:0/571653447 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4268031070 con 0x7f42781033a0 2024-04-25T00:21:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4260039150 msgr2=0x7f426003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4260039150 0x7f426003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f426c006fd0 tx=0x7f426c006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 msgr2=0x7f4278075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f4278075ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4268005800 tx=0x7f4268009f60 comp rx=0 tx=0).stop 2024-04-25T00:21:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 shutdown_connections 2024-04-25T00:21:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4260039150 0x7f426003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 --2- 172.21.15.31:0/571653447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42781033a0 0x7f4278075ef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.701+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 >> 172.21.15.31:0/571653447 conn(0x7f42780fd280 msgr2=0x7f42780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.702+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 shutdown_connections 2024-04-25T00:21:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:25.702+0000 7f427c8b8700 1 -- 172.21.15.31:0/571653447 wait complete. 2024-04-25T00:21:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:21:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:26.288 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T00:21:26.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T00:21:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:26.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:26 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/571653447' entity='client.admin' 2024-04-25T00:21:26.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:26.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 -- 172.21.15.31:0/2614609289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 msgr2=0x7fce80101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 --2- 172.21.15.31:0/2614609289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce80101000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce70009910 tx=0x7fce70009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 -- 172.21.15.31:0/2614609289 shutdown_connections 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 --2- 172.21.15.31:0/2614609289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce80101000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 -- 172.21.15.31:0/2614609289 >> 172.21.15.31:0/2614609289 conn(0x7fce800fc140 msgr2=0x7fce800fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 -- 172.21.15.31:0/2614609289 shutdown_connections 2024-04-25T00:21:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.091+0000 7fce84f90700 1 -- 172.21.15.31:0/2614609289 wait complete. 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce84f90700 1 Processor -- start 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce84f90700 1 -- start start 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce84f90700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce801994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce84f90700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce80106b20 con 0x7fce80100be0 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce7e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce801994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce7e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce801994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40864/0 (socket says 172.21.15.31:40864) 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce7e59c700 1 -- 172.21.15.31:0/4069249399 learned_addr learned my addr 172.21.15.31:0/4069249399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.092+0000 7fce7e59c700 1 -- 172.21.15.31:0/4069249399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce80199a30 con 0x7fce80100be0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.093+0000 7fce7e59c700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce801994f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce70005800 tx=0x7fce70009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.093+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce70027070 con 0x7fce80100be0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.093+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7000be10 con 0x7fce80100be0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.093+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce80199cc0 con 0x7fce80100be0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.093+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce7000f460 con 0x7fce80100be0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.093+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8019c9b0 con 0x7fce80100be0 2024-04-25T00:21:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.094+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fce7002a070 con 0x7fce80100be0 2024-04-25T00:21:28.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.094+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce801072a0 con 0x7fce80100be0 2024-04-25T00:21:28.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.094+0000 7fce6f7fe700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce680390f0 0x7fce6803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.094+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce7001e030 con 0x7fce80100be0 2024-04-25T00:21:28.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.094+0000 7fce7dd9b700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce680390f0 0x7fce6803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.094+0000 7fce7dd9b700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce680390f0 0x7fce6803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce74006fd0 tx=0x7fce74006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.099+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce70013070 con 0x7fce80100be0 2024-04-25T00:21:28.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.309+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fce8002dc40 con 0x7fce680390f0 2024-04-25T00:21:28.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.312+0000 7fce6f7fe700 1 -- 172.21.15.31:0/4069249399 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce8002dc40 con 0x7fce680390f0 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce680390f0 msgr2=0x7fce6803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce680390f0 0x7fce6803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce74006fd0 tx=0x7fce74006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 msgr2=0x7fce801994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce801994f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce70005800 tx=0x7fce70009f60 comp rx=0 tx=0).stop 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 shutdown_connections 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce680390f0 0x7fce6803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 --2- 172.21.15.31:0/4069249399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce80100be0 0x7fce801994f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.314+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 >> 172.21.15.31:0/4069249399 conn(0x7fce800fc140 msgr2=0x7fce801905d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.315+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 shutdown_connections 2024-04-25T00:21:28.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:28.315+0000 7fce84f90700 1 -- 172.21.15.31:0/4069249399 wait complete. 2024-04-25T00:21:29.080 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi040 2024-04-25T00:21:29.080 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:21:29.080 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:21:29.099 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:21:29.099 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:29.157 INFO:tasks.cephadm:Adding host smithi040 to orchestrator... 2024-04-25T00:21:29.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch host add smithi040 2024-04-25T00:21:29.406 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:29.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:29.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:29.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:29.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:29.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:29.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T00:21:29.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: Updating smithi031:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:21:29.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:29.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: Updating smithi031:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:21:29.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:29.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:29.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.830+0000 7f180fd16700 1 -- 172.21.15.31:0/3112151549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 msgr2=0x7f1808102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.830+0000 7f180fd16700 1 --2- 172.21.15.31:0/3112151549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808102280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1804009910 tx=0x7f1804009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.830+0000 7f180fd16700 1 -- 172.21.15.31:0/3112151549 shutdown_connections 2024-04-25T00:21:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.830+0000 7f180fd16700 1 --2- 172.21.15.31:0/3112151549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808102280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.830+0000 7f180fd16700 1 -- 172.21.15.31:0/3112151549 >> 172.21.15.31:0/3112151549 conn(0x7f18080fd770 msgr2=0x7f18080ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180fd16700 1 -- 172.21.15.31:0/3112151549 shutdown_connections 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180fd16700 1 -- 172.21.15.31:0/3112151549 wait complete. 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180fd16700 1 Processor -- start 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180fd16700 1 -- start start 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180fd16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180fd16700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f180810c120 con 0x7f1808101ea0 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.831+0000 7f180dab2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f180dab2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40890/0 (socket says 172.21.15.31:40890) 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f180dab2700 1 -- 172.21.15.31:0/1219420479 learned_addr learned my addr 172.21.15.31:0/1219420479 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f180dab2700 1 -- 172.21.15.31:0/1219420479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1808076420 con 0x7f1808101ea0 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f180dab2700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808074530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1804006010 tx=0x7f1804005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1804026070 con 0x7f1808101ea0 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1808074ad0 con 0x7f1808101ea0 2024-04-25T00:21:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f180400bc00 con 0x7f1808101ea0 2024-04-25T00:21:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1804003930 con 0x7f1808101ea0 2024-04-25T00:21:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.832+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1808074f90 con 0x7f1808101ea0 2024-04-25T00:21:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.833+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1804029070 con 0x7f1808101ea0 2024-04-25T00:21:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.833+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1808103380 con 0x7f1808101ea0 2024-04-25T00:21:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.833+0000 7f17feffd700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f17f40390f0 0x7f17f403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.834+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f180401e070 con 0x7f1808101ea0 2024-04-25T00:21:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.834+0000 7f180d2b1700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f17f40390f0 0x7f17f403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.834+0000 7f180d2b1700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f17f40390f0 0x7f17f403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f17f8006fd0 tx=0x7f17f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:30.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:30.838+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1804013070 con 0x7f1808101ea0 2024-04-25T00:21:31.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:31.049+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}) -- 0x7f1808075220 con 0x7f17f40390f0 2024-04-25T00:21:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:31 smithi031 ceph-mon[29167]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:32.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:32 smithi031 ceph-mon[29167]: Deploying cephadm binary to smithi040 2024-04-25T00:21:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:34 smithi031 ceph-mon[29167]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:35.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.302+0000 7f17feffd700 1 -- 172.21.15.31:0/1219420479 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1808075220 con 0x7f17f40390f0 2024-04-25T00:21:35.303 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi040' with addr '172.21.15.40' 2024-04-25T00:21:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f17f40390f0 msgr2=0x7f17f403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f17f40390f0 0x7f17f403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f17f8006fd0 tx=0x7f17f8006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 msgr2=0x7f1808074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808074530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1804006010 tx=0x7f1804005780 comp rx=0 tx=0).stop 2024-04-25T00:21:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 shutdown_connections 2024-04-25T00:21:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f17f40390f0 0x7f17f403b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 --2- 172.21.15.31:0/1219420479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1808101ea0 0x7f1808074530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 >> 172.21.15.31:0/1219420479 conn(0x7f18080fd770 msgr2=0x7f1808103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 shutdown_connections 2024-04-25T00:21:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:35.304+0000 7f180fd16700 1 -- 172.21.15.31:0/1219420479 wait complete. 2024-04-25T00:21:35.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:21:36.164 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:36.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:36 smithi031 ceph-mon[29167]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:36.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:36.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:36 smithi031 ceph-mon[29167]: Added host smithi040 2024-04-25T00:21:36.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:36.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:37.621 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:37 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.726+0000 7fa9b0719700 1 -- 172.21.15.31:0/3453896107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 msgr2=0x7fa9a81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.726+0000 7fa9b0719700 1 --2- 172.21.15.31:0/3453896107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81011c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009910 tx=0x7fa9a4009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.726+0000 7fa9b0719700 1 -- 172.21.15.31:0/3453896107 shutdown_connections 2024-04-25T00:21:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.726+0000 7fa9b0719700 1 --2- 172.21.15.31:0/3453896107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81011c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.726+0000 7fa9b0719700 1 -- 172.21.15.31:0/3453896107 >> 172.21.15.31:0/3453896107 conn(0x7fa9a80fc690 msgr2=0x7fa9a80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9b0719700 1 -- 172.21.15.31:0/3453896107 shutdown_connections 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9b0719700 1 -- 172.21.15.31:0/3453896107 wait complete. 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9b0719700 1 Processor -- start 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9b0719700 1 -- start start 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9b0719700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9b0719700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a810b060 con 0x7fa9a8100de0 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9ae4b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9ae4b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33446/0 (socket says 172.21.15.31:33446) 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.727+0000 7fa9ae4b5700 1 -- 172.21.15.31:0/2852022670 learned_addr learned my addr 172.21.15.31:0/2852022670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa9ae4b5700 1 -- 172.21.15.31:0/2852022670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a8196b00 con 0x7fa9a8100de0 2024-04-25T00:21:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa9ae4b5700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81965c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40056d0 tx=0x7fa9a40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4026070 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a400bba0 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a8196df0 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a40037f0 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.728+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a819ae90 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.729+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa9a4029070 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.729+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a81022c0 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.729+0000 7fa99f7fe700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9940390f0 0x7fa99403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.729+0000 7fa9adcb4700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9940390f0 0x7fa99403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.729+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa9a401e070 con 0x7fa9a8100de0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.729+0000 7fa9adcb4700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9940390f0 0x7fa99403b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa998006fd0 tx=0x7fa998006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.734+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa9a4013070 con 0x7fa9a8100de0 2024-04-25T00:21:37.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.945+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9a8197920 con 0x7fa9940390f0 2024-04-25T00:21:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.947+0000 7fa99f7fe700 1 -- 172.21.15.31:0/2852022670 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fa9a8197920 con 0x7fa9940390f0 2024-04-25T00:21:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:37.948 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}] 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9940390f0 msgr2=0x7fa99403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9940390f0 0x7fa99403b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa998006fd0 tx=0x7fa998006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 msgr2=0x7fa9a81965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81965c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40056d0 tx=0x7fa9a40057b0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 shutdown_connections 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9940390f0 0x7fa99403b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 --2- 172.21.15.31:0/2852022670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9a8100de0 0x7fa9a81965c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 >> 172.21.15.31:0/2852022670 conn(0x7fa9a80fc690 msgr2=0x7fa9a81027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 shutdown_connections 2024-04-25T00:21:37.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:37.949+0000 7fa9b0719700 1 -- 172.21.15.31:0/2852022670 wait complete. 2024-04-25T00:21:38.578 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-04-25T00:21:38.578 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:21:38.579 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:21:38.597 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:21:38.597 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:38.654 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-04-25T00:21:38.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch host add smithi169 2024-04-25T00:21:38.901 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:39.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:38 smithi031 ceph-mon[29167]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:39.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:38 smithi031 ceph-mon[29167]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:39.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:40.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 -- 172.21.15.31:0/446279217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 msgr2=0x7fa098069a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 --2- 172.21.15.31:0/446279217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa098069a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa088009910 tx=0x7fa088009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 -- 172.21.15.31:0/446279217 shutdown_connections 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 --2- 172.21.15.31:0/446279217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa098069a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 -- 172.21.15.31:0/446279217 >> 172.21.15.31:0/446279217 conn(0x7fa098076690 msgr2=0x7fa098076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 -- 172.21.15.31:0/446279217 shutdown_connections 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.279+0000 7fa09d3d1700 1 -- 172.21.15.31:0/446279217 wait complete. 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa09d3d1700 1 Processor -- start 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa09d3d1700 1 -- start start 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa09d3d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa09819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa09d3d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa09810e380 con 0x7fa098069650 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa096ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa09819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:40.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa096ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa09819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33476/0 (socket says 172.21.15.31:33476) 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa096ffd700 1 -- 172.21.15.31:0/1610149425 learned_addr learned my addr 172.21.15.31:0/1610149425 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.280+0000 7fa096ffd700 1 -- 172.21.15.31:0/1610149425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09819fb90 con 0x7fa098069650 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.281+0000 7fa096ffd700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa09819f650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0880056d0 tx=0x7fa0880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.281+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa088026070 con 0x7fa098069650 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.281+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa08800bc00 con 0x7fa098069650 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.281+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0880039b0 con 0x7fa098069650 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.281+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09819fe20 con 0x7fa098069650 2024-04-25T00:21:40.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.281+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa098199720 con 0x7fa098069650 2024-04-25T00:21:40.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.282+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa088029070 con 0x7fa098069650 2024-04-25T00:21:40.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.282+0000 7fa07ffff700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa0800390f0 0x7fa08003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:40.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.282+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa08801e070 con 0x7fa098069650 2024-04-25T00:21:40.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.282+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa098101160 con 0x7fa098069650 2024-04-25T00:21:40.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.282+0000 7fa0967fc700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa0800390f0 0x7fa08003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:40.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.282+0000 7fa0967fc700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa0800390f0 0x7fa08003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa08c006fd0 tx=0x7fa08c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:40.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.287+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa088030070 con 0x7fa098069650 2024-04-25T00:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:40.500+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7fa098063890 con 0x7fa0800390f0 2024-04-25T00:21:41.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:41 smithi031 ceph-mon[29167]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: Deploying cephadm binary to smithi169 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:42.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:43.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:43 smithi031 ceph-mon[29167]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:43.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:43 smithi031 ceph-mon[29167]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T00:21:43.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:43 smithi031 ceph-mon[29167]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:21:44.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.773+0000 7fa07ffff700 1 -- 172.21.15.31:0/1610149425 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa098063890 con 0x7fa0800390f0 2024-04-25T00:21:44.774 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa0800390f0 msgr2=0x7fa08003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa0800390f0 0x7fa08003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa08c006fd0 tx=0x7fa08c006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 msgr2=0x7fa09819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa09819f650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0880056d0 tx=0x7fa0880057b0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 shutdown_connections 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa0800390f0 0x7fa08003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 --2- 172.21.15.31:0/1610149425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098069650 0x7fa09819f650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 >> 172.21.15.31:0/1610149425 conn(0x7fa098076690 msgr2=0x7fa0981091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 shutdown_connections 2024-04-25T00:21:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:44.775+0000 7fa09d3d1700 1 -- 172.21.15.31:0/1610149425 wait complete. 2024-04-25T00:21:45.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:45 smithi031 ceph-mon[29167]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:45.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:45.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:45.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:45.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:45.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:45.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:21:45.706 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:46.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:46 smithi031 ceph-mon[29167]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:46.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:46 smithi031 ceph-mon[29167]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:21:46.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:46 smithi031 ceph-mon[29167]: Added host smithi169 2024-04-25T00:21:46.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:47.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:47 smithi031 ceph-mon[29167]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:47.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 -- 172.21.15.31:0/1521873114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 msgr2=0x7f67d0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 --2- 172.21.15.31:0/1521873114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d0102280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009910 tx=0x7f67c0009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 -- 172.21.15.31:0/1521873114 shutdown_connections 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 --2- 172.21.15.31:0/1521873114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d0102280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 -- 172.21.15.31:0/1521873114 >> 172.21.15.31:0/1521873114 conn(0x7f67d00fd770 msgr2=0x7f67d00ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 -- 172.21.15.31:0/1521873114 shutdown_connections 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.240+0000 7f67d7b3b700 1 -- 172.21.15.31:0/1521873114 wait complete. 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d7b3b700 1 Processor -- start 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d7b3b700 1 -- start start 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d7b3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d01976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d7b3b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67d010c120 con 0x7f67d0101ea0 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d58d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d01976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d58d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d01976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44630/0 (socket says 172.21.15.31:44630) 2024-04-25T00:21:47.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d58d7700 1 -- 172.21.15.31:0/2432493152 learned_addr learned my addr 172.21.15.31:0/2432493152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.241+0000 7f67d58d7700 1 -- 172.21.15.31:0/2432493152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d0197bf0 con 0x7f67d0101ea0 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.242+0000 7f67d58d7700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d01976b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67c0006010 tx=0x7f67c0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.242+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c0026070 con 0x7f67d0101ea0 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.242+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67c000bba0 con 0x7f67d0101ea0 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.242+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d0197ee0 con 0x7f67d0101ea0 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.242+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c00038a0 con 0x7f67d0101ea0 2024-04-25T00:21:47.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.242+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d019bf80 con 0x7f67d0101ea0 2024-04-25T00:21:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.243+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f67c0029070 con 0x7f67d0101ea0 2024-04-25T00:21:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.243+0000 7f67c6ffd700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67bc039150 0x7f67bc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.243+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d0103380 con 0x7f67d0101ea0 2024-04-25T00:21:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.243+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f67c001e070 con 0x7f67d0101ea0 2024-04-25T00:21:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.243+0000 7f67d50d6700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67bc039150 0x7f67bc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.243+0000 7f67d50d6700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67bc039150 0x7f67bc03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006fd0 tx=0x7f67cc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.248+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67c0013070 con 0x7f67d0101ea0 2024-04-25T00:21:47.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.459+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67d0198a10 con 0x7f67bc039150 2024-04-25T00:21:47.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.461+0000 7f67c6ffd700 1 -- 172.21.15.31:0/2432493152 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f67d0198a10 con 0x7f67bc039150 2024-04-25T00:21:47.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:21:47.461 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-04-25T00:21:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.462+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67bc039150 msgr2=0x7f67bc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67bc039150 0x7f67bc03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006fd0 tx=0x7f67cc006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 msgr2=0x7f67d01976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d01976b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67c0006010 tx=0x7f67c0005780 comp rx=0 tx=0).stop 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 shutdown_connections 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67bc039150 0x7f67bc03b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 --2- 172.21.15.31:0/2432493152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d0101ea0 0x7f67d01976b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 >> 172.21.15.31:0/2432493152 conn(0x7f67d00fd770 msgr2=0x7f67d0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 shutdown_connections 2024-04-25T00:21:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:47.463+0000 7f67d7b3b700 1 -- 172.21.15.31:0/2432493152 wait complete. 2024-04-25T00:21:48.088 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T00:21:48.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T00:21:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:48.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:48 smithi031 ceph-mon[29167]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:48.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:48 smithi031 ceph-mon[29167]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:48.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/3974547218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 msgr2=0x7f9edc107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 --2- 172.21.15.31:0/3974547218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc107060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009910 tx=0x7f9ecc009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/3974547218 shutdown_connections 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 --2- 172.21.15.31:0/3974547218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc107060 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/3974547218 >> 172.21.15.31:0/3974547218 conn(0x7f9edc0fb9a0 msgr2=0x7f9edc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/3974547218 shutdown_connections 2024-04-25T00:21:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/3974547218 wait complete. 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 Processor -- start 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.789+0000 7f9ee2aa5700 1 -- start start 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee2aa5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee2aa5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10c1f0 con 0x7f9edc106c80 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee0841700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee0841700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44658/0 (socket says 172.21.15.31:44658) 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee0841700 1 -- 172.21.15.31:0/2606705163 learned_addr learned my addr 172.21.15.31:0/2606705163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee0841700 1 -- 172.21.15.31:0/2606705163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc19fb30 con 0x7f9edc106c80 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee0841700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc19f5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0056d0 tx=0x7f9ecc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ecc026070 con 0x7f9edc106c80 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ecc00bc00 con 0x7f9edc106c80 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc19fdc0 con 0x7f9edc106c80 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.790+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ecc0039b0 con 0x7f9edc106c80 2024-04-25T00:21:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.791+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc1996c0 con 0x7f9edc106c80 2024-04-25T00:21:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.791+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9ecc029070 con 0x7f9edc106c80 2024-04-25T00:21:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.792+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc101170 con 0x7f9edc106c80 2024-04-25T00:21:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.792+0000 7f9ed9ffb700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ec40390f0 0x7f9ec403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.792+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ecc01e070 con 0x7f9edc106c80 2024-04-25T00:21:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.792+0000 7f9edbfff700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ec40390f0 0x7f9ec403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.792+0000 7f9edbfff700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ec40390f0 0x7f9ec403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006fd0 tx=0x7f9ed0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:49.797+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ecc013070 con 0x7f9edc106c80 2024-04-25T00:21:50.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.008+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9edc19abf0 con 0x7f9edc106c80 2024-04-25T00:21:50.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.477+0000 7f9ed9ffb700 1 -- 172.21.15.31:0/2606705163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9ecc030090 con 0x7f9edc106c80 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.480+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ec40390f0 msgr2=0x7f9ec403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.480+0000 7f9ee2aa5700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ec40390f0 0x7f9ec403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006fd0 tx=0x7f9ed0006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.480+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 msgr2=0x7f9edc19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.480+0000 7f9ee2aa5700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc19f5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0056d0 tx=0x7f9ecc0057b0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.481+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 shutdown_connections 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.481+0000 7f9ee2aa5700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ec40390f0 0x7f9ec403b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.481+0000 7f9ee2aa5700 1 --2- 172.21.15.31:0/2606705163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9edc106c80 0x7f9edc19f5f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.481+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 >> 172.21.15.31:0/2606705163 conn(0x7f9edc0fb9a0 msgr2=0x7f9edc109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.481+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 shutdown_connections 2024-04-25T00:21:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:21:50.481+0000 7f9ee2aa5700 1 -- 172.21.15.31:0/2606705163 wait complete. 2024-04-25T00:21:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-04-25T00:21:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:50 smithi031 ceph-mon[29167]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:50 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2606705163' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:21:51.132 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-04-25T00:21:51.132 INFO:tasks.cephadm:Adding mon.b on smithi040 2024-04-25T00:21:51.132 INFO:tasks.cephadm:Adding mon.c on smithi169 2024-04-25T00:21:51.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c' 2024-04-25T00:21:51.368 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2606705163' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:21:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:51.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:51.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T00:21:51.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:21:51.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:51 smithi031 ceph-mon[29167]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:52 smithi031 ceph-mon[29167]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:52 smithi031 ceph-mon[29167]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:21:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:52.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 -- 172.21.15.169:0/736270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 msgr2=0x7fa134100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:52.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 --2- 172.21.15.169:0/736270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa134100fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa124009910 tx=0x7fa124009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:52.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 -- 172.21.15.169:0/736270747 shutdown_connections 2024-04-25T00:21:52.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 --2- 172.21.15.169:0/736270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa134100fd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 -- 172.21.15.169:0/736270747 >> 172.21.15.169:0/736270747 conn(0x7fa1340fc630 msgr2=0x7fa1340fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:52.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 -- 172.21.15.169:0/736270747 shutdown_connections 2024-04-25T00:21:52.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.879+0000 7fa13bcbe700 1 -- 172.21.15.169:0/736270747 wait complete. 2024-04-25T00:21:52.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.880+0000 7fa13bcbe700 1 Processor -- start 2024-04-25T00:21:52.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.880+0000 7fa13bcbe700 1 -- start start 2024-04-25T00:21:52.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.880+0000 7fa13bcbe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa13419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.880+0000 7fa13bcbe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341127a0 con 0x7fa134100bf0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.880+0000 7fa139a5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa13419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa139a5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa13419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.169:46836/0 (socket says 172.21.15.169:46836) 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa139a5a700 1 -- 172.21.15.169:0/3073560815 learned_addr learned my addr 172.21.15.169:0/3073560815 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa139a5a700 1 -- 172.21.15.169:0/3073560815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13419e250 con 0x7fa134100bf0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa139a5a700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa13419dd10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1240056d0 tx=0x7fa1240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124026070 con 0x7fa134100bf0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa124004370 con 0x7fa134100bf0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13419e4e0 con 0x7fa134100bf0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1240039b0 con 0x7fa134100bf0 2024-04-25T00:21:52.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.881+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13419e940 con 0x7fa134100bf0 2024-04-25T00:21:52.884 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.882+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13406a980 con 0x7fa134100bf0 2024-04-25T00:21:52.884 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.882+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa124029070 con 0x7fa134100bf0 2024-04-25T00:21:52.884 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.882+0000 7fa12affd700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa1200390f0 0x7fa12003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:52.884 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.883+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa12401e070 con 0x7fa134100bf0 2024-04-25T00:21:52.884 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.883+0000 7fa139259700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa1200390f0 0x7fa12003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:52.885 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.883+0000 7fa139259700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa1200390f0 0x7fa12003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa130006fd0 tx=0x7fa130006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:52.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:52.888+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa124013070 con 0x7fa134100bf0 2024-04-25T00:21:53.092 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.091+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}) -- 0x7fa1340fe5e0 con 0x7fa1200390f0 2024-04-25T00:21:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.097+0000 7fa12affd700 1 -- 172.21.15.169:0/3073560815 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa1340fe5e0 con 0x7fa1200390f0 2024-04-25T00:21:53.098 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa1200390f0 msgr2=0x7fa12003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa1200390f0 0x7fa12003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa130006fd0 tx=0x7fa130006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 msgr2=0x7fa13419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa13419dd10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1240056d0 tx=0x7fa1240057b0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 shutdown_connections 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa1200390f0 0x7fa12003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 --2- 172.21.15.169:0/3073560815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa134100bf0 0x7fa13419dd10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 >> 172.21.15.169:0/3073560815 conn(0x7fa1340fc630 msgr2=0x7fa1340fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 shutdown_connections 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:53.100+0000 7fa13bcbe700 1 -- 172.21.15.169:0/3073560815 wait complete. 2024-04-25T00:21:53.654 DEBUG:teuthology.orchestra.run.smithi040:mon.b> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T00:21:53.656 DEBUG:teuthology.orchestra.run.smithi169:mon.c> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T00:21:53.661 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:21:53.661 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:21:54.379 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c;count:3 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:21:54.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:21:54.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:55 smithi031 ceph-mon[29167]: Deploying daemon mon.c on smithi169 2024-04-25T00:21:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:55 smithi031 ceph-mon[29167]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:57 smithi031 ceph-mon[29167]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:21:57.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2715346587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 msgr2=0x7f4dfc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 --2- 172.21.15.169:0/2715346587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc102280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009910 tx=0x7f4df8009c20 comp rx=0 tx=0).stop 2024-04-25T00:21:57.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2715346587 shutdown_connections 2024-04-25T00:21:57.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 --2- 172.21.15.169:0/2715346587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc102280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2715346587 >> 172.21.15.169:0/2715346587 conn(0x7f4dfc0fd770 msgr2=0x7f4dfc0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2715346587 shutdown_connections 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2715346587 wait complete. 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 Processor -- start 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.526+0000 7f4e03bc7700 1 -- start start 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e03bc7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e03bc7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc10c120 con 0x7f4dfc101ea0 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e01963700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e01963700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc1976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.169:46864/0 (socket says 172.21.15.169:46864) 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e01963700 1 -- 172.21.15.169:0/2593487859 learned_addr learned my addr 172.21.15.169:0/2593487859 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e01963700 1 -- 172.21.15.169:0/2593487859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc197bf0 con 0x7f4dfc101ea0 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.527+0000 7f4e01963700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc1976b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4df80056d0 tx=0x7f4df80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4df8026070 con 0x7f4dfc101ea0 2024-04-25T00:21:57.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df800bba0 con 0x7f4dfc101ea0 2024-04-25T00:21:57.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc197ee0 con 0x7f4dfc101ea0 2024-04-25T00:21:57.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4df80037f0 con 0x7f4dfc101ea0 2024-04-25T00:21:57.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc19bf80 con 0x7f4dfc101ea0 2024-04-25T00:21:57.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4df8029070 con 0x7f4dfc101ea0 2024-04-25T00:21:57.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.528+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc103380 con 0x7f4dfc101ea0 2024-04-25T00:21:57.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.529+0000 7f4df2ffd700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4de8039150 0x7f4de803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.529+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4df801e070 con 0x7f4dfc101ea0 2024-04-25T00:21:57.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.529+0000 7f4e01162700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4de8039150 0x7f4de803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.529+0000 7f4e01162700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4de8039150 0x7f4de803b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006fd0 tx=0x7f4dec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.534+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4df8013070 con 0x7f4dfc101ea0 2024-04-25T00:21:57.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.833+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4dfc198a10 con 0x7f4dfc101ea0 2024-04-25T00:21:57.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.834+0000 7f4df2ffd700 1 -- 172.21.15.169:0/2593487859 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4df8030020 con 0x7f4dfc101ea0 2024-04-25T00:21:57.835 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:21:57.835 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","modified":"2024-04-25T00:20:09.931607Z","created":"2024-04-25T00:20:09.931607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4de8039150 msgr2=0x7f4de803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4de8039150 0x7f4de803b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006fd0 tx=0x7f4dec006e40 comp rx=0 tx=0).stop 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 msgr2=0x7f4dfc1976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc1976b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4df80056d0 tx=0x7f4df80057b0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 shutdown_connections 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4de8039150 0x7f4de803b610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 --2- 172.21.15.169:0/2593487859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc101ea0 0x7f4dfc1976b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 >> 172.21.15.169:0/2593487859 conn(0x7f4dfc0fd770 msgr2=0x7f4dfc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 shutdown_connections 2024-04-25T00:21:57.838 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:21:57.836+0000 7f4e03bc7700 1 -- 172.21.15.169:0/2593487859 wait complete. 2024-04-25T00:21:57.838 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-04-25T00:21:58.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:21:58 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/2593487859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:21:58.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:21:58 smithi169 ceph-mon[33112]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:21:59.383 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:21:59.383 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:21:59.617 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:22:03.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:22:03.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: mon.a calling monitor election 2024-04-25T00:22:03.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: mon.c calling monitor election 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: monmap epoch 2 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: last_changed 2024-04-25T00:21:58.550773+0000 2024-04-25T00:22:03.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: min_mon_release 19 (squid) 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: election_strategy: 1 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: fsmap 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: mgrmap e13: x(active, since 50s) 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: overall HEALTH_OK 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:03.895 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:03.896 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: mon.a calling monitor election 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: mon.c calling monitor election 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: monmap epoch 2 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: last_changed 2024-04-25T00:21:58.550773+0000 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: min_mon_release 19 (squid) 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: election_strategy: 1 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: fsmap 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: mgrmap e13: x(active, since 50s) 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: overall HEALTH_OK 2024-04-25T00:22:03.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:03.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:03.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:03.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:03.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:04.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 -- 172.21.15.169:0/3270343072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63e80035c0 msgr2=0x7f63e8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 --2- 172.21.15.169:0/3270343072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63e80035c0 0x7f63e8005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63fc101c70 tx=0x7f63ec00feb0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 -- 172.21.15.169:0/3270343072 shutdown_connections 2024-04-25T00:22:04.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 --2- 172.21.15.169:0/3270343072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63e80035c0 0x7f63e8005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 -- 172.21.15.169:0/3270343072 >> 172.21.15.169:0/3270343072 conn(0x7f63fc0fd830 msgr2=0x7f63fc0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:04.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 -- 172.21.15.169:0/3270343072 shutdown_connections 2024-04-25T00:22:04.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.103+0000 7f6401fa5700 1 -- 172.21.15.169:0/3270343072 wait complete. 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.104+0000 7f6401fa5700 1 Processor -- start 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.104+0000 7f6401fa5700 1 -- start start 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f6401fa5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f6401fa5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63fc074a70 0x7f63fc074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f6401fa5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc109c80 con 0x7f63fc074a70 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f6401fa5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc109b00 con 0x7f63e80035c0 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55020/0 (socket says 172.21.15.169:55020) 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 -- 172.21.15.169:0/2708331445 learned_addr learned my addr 172.21.15.169:0/2708331445 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63f2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63fc074a70 0x7f63fc074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 -- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 msgr2=0x7f63fc074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 -- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 msgr2=0x7f63fc074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:22:04.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63fb7fe700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63f2ffd700 1 -- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 msgr2=0x7f63fc074530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63f2ffd700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.105+0000 7f63f2ffd700 1 -- 172.21.15.169:0/2708331445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc076450 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.106+0000 7f63f2ffd700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63fc074a70 0x7f63fc074ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b4d0 tx=0x7f63e400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.106+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f63e4011630 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.106+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63e4011c80 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.106+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1a3aa0 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.106+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f63e4010da0 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.106+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a3fe0 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.107+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f63e4018020 con 0x7f63fc074a70 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.107+0000 7f63f97fa700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f63dc038ee0 0x7f63dc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.107+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f63e404c4d0 con 0x7f63fc074a70 2024-04-25T00:22:04.112 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.107+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc103480 con 0x7f63fc074a70 2024-04-25T00:22:04.112 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.107+0000 7f63fb7fe700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f63dc038ee0 0x7f63dc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.112 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.108+0000 7f63fb7fe700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f63dc038ee0 0x7f63dc03b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00fa60 tx=0x7f63ec005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:04.113 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.113+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63e401f020 con 0x7f63fc074a70 2024-04-25T00:22:04.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.397+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f63fc02db30 con 0x7f63fc074a70 2024-04-25T00:22:04.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.397+0000 7f63f97fa700 1 -- 172.21.15.169:0/2708331445 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f63e40193b0 con 0x7f63fc074a70 2024-04-25T00:22:04.398 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:22:04.398 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","modified":"2024-04-25T00:21:58.550773Z","created":"2024-04-25T00:20:09.931607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:22:04.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f63dc038ee0 msgr2=0x7f63dc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f63dc038ee0 0x7f63dc03b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00fa60 tx=0x7f63ec005a90 comp rx=0 tx=0).stop 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63fc074a70 msgr2=0x7f63fc074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63fc074a70 0x7f63fc074ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b4d0 tx=0x7f63e400b890 comp rx=0 tx=0).stop 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 shutdown_connections 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e80035c0 0x7f63fc074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f63dc038ee0 0x7f63dc03b3a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 --2- 172.21.15.169:0/2708331445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63fc074a70 0x7f63fc074ef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 >> 172.21.15.169:0/2708331445 conn(0x7f63fc0fd830 msgr2=0x7f63fc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 shutdown_connections 2024-04-25T00:22:04.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:04.400+0000 7f6401fa5700 1 -- 172.21.15.169:0/2708331445 wait complete. 2024-04-25T00:22:04.402 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-04-25T00:22:04.720 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:04 smithi169 ceph-mon[33112]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:04.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:04 smithi169 ceph-mon[33112]: Deploying daemon mon.b on smithi040 2024-04-25T00:22:04.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:04 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/2708331445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:04.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:04 smithi031 ceph-mon[29167]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:04 smithi031 ceph-mon[29167]: Deploying daemon mon.b on smithi040 2024-04-25T00:22:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:04 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/2708331445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:06.015 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:22:06.015 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:22:06.257 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:22:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:06 smithi169 ceph-mon[33112]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:06 smithi031 ceph-mon[29167]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.575 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 -- 172.21.15.169:0/1179793151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b8100ea0 msgr2=0x7fb3b8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.575 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/1179793151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b8100ea0 0x7fb3b8101280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009910 tx=0x7fb3b0009c20 comp rx=0 tx=0).stop 2024-04-25T00:22:07.575 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 -- 172.21.15.169:0/1179793151 shutdown_connections 2024-04-25T00:22:07.575 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/1179793151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b8100ea0 0x7fb3b8101280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.575 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 -- 172.21.15.169:0/1179793151 >> 172.21.15.169:0/1179793151 conn(0x7fb3b80fc770 msgr2=0x7fb3b80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.575 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 -- 172.21.15.169:0/1179793151 shutdown_connections 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 -- 172.21.15.169:0/1179793151 wait complete. 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.573+0000 7fb3bfd75700 1 Processor -- start 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bfd75700 1 -- start start 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bfd75700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bfd75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b819dba0 0x7fb3b8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bfd75700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b810b120 con 0x7fb3b819dba0 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bfd75700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b810afa0 con 0x7fb3b8100ea0 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55034/0 (socket says 172.21.15.169:55034) 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 -- 172.21.15.169:0/3116463821 learned_addr learned my addr 172.21.15.169:0/3116463821 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 -- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 msgr2=0x7fb3b819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 -- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 msgr2=0x7fb3b819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:22:07.576 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:22:07.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bdb11700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:22:07.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.574+0000 7fb3bd310700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b819dba0 0x7fb3b8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3bd310700 1 -- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 msgr2=0x7fb3b819d660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:22:07.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3bd310700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3bd310700 1 -- 172.21.15.169:0/3116463821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b819e120 con 0x7fb3b819dba0 2024-04-25T00:22:07.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3bd310700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b819dba0 0x7fb3b8197730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b4d0 tx=0x7fb3a800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.578 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb3a8011630 con 0x7fb3b819dba0 2024-04-25T00:22:07.578 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b8197cd0 con 0x7fb3b819dba0 2024-04-25T00:22:07.578 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a8011c80 con 0x7fb3b819dba0 2024-04-25T00:22:07.578 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb3a8010da0 con 0x7fb3b819dba0 2024-04-25T00:22:07.578 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.575+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b8198260 con 0x7fb3b819dba0 2024-04-25T00:22:07.579 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.576+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb3a8018020 con 0x7fb3b819dba0 2024-04-25T00:22:07.579 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.576+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b8102380 con 0x7fb3b819dba0 2024-04-25T00:22:07.579 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.576+0000 7fb3aeffd700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb3a4038e90 0x7fb3a403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.579 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.576+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb3a804bcd0 con 0x7fb3b819dba0 2024-04-25T00:22:07.579 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.576+0000 7fb3bdb11700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb3a4038e90 0x7fb3a403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.579 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.577+0000 7fb3bdb11700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb3a4038e90 0x7fb3a403b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009750 tx=0x7fb3b001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.582 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.581+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3a801f020 con 0x7fb3b819dba0 2024-04-25T00:22:07.629 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 podman[32918]: 2024-04-25 00:22:07.524131654 +0000 UTC m=+0.021317685 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:22:07.855 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.855+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb3b8198510 con 0x7fb3b819dba0 2024-04-25T00:22:07.856 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.855+0000 7fb3aeffd700 1 -- 172.21.15.169:0/3116463821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb3a801a920 con 0x7fb3b819dba0 2024-04-25T00:22:07.856 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:22:07.856 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","modified":"2024-04-25T00:21:58.550773Z","created":"2024-04-25T00:20:09.931607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:22:07.858 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb3a4038e90 msgr2=0x7fb3a403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.858 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb3a4038e90 0x7fb3a403b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009750 tx=0x7fb3b001a040 comp rx=0 tx=0).stop 2024-04-25T00:22:07.858 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b819dba0 msgr2=0x7fb3b8197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b819dba0 0x7fb3b8197730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b4d0 tx=0x7fb3a800b890 comp rx=0 tx=0).stop 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 shutdown_connections 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b8100ea0 0x7fb3b819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb3a4038e90 0x7fb3a403b350 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 --2- 172.21.15.169:0/3116463821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b819dba0 0x7fb3b8197730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.857+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 >> 172.21.15.169:0/3116463821 conn(0x7fb3b80fc770 msgr2=0x7fb3b806a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.858+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 shutdown_connections 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:07.858+0000 7fb3bfd75700 1 -- 172.21.15.169:0/3116463821 wait complete. 2024-04-25T00:22:07.859 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 podman[32918]: 2024-04-25 00:22:07.834613126 +0000 UTC m=+0.331799137 container init b28e66123773c25c33fff9905cb17391123d4860c0cd80972342982414b63d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 podman[32918]: 2024-04-25 00:22:07.83798776 +0000 UTC m=+0.335173771 container start b28e66123773c25c33fff9905cb17391123d4860c0cd80972342982414b63d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pidfile_write: ignore empty --pid-file 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: load: jerasure load: lrc 2024-04-25T00:22:07.889 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Git sha 0 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: DB SUMMARY 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: DB Session ID: TODYHMTQ1MNGU3HF9YI5 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:22:07.890 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:22:07.891 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:22:07.891 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.env: 0x55f2d02f5c60 2024-04-25T00:22:07.891 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:22:07.891 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.info_log: 0x55f2d1e7daa0 2024-04-25T00:22:07.891 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.statistics: (nil) 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.use_fsync: 0 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:22:07.893 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.db_log_dir: 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.wal_dir: 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:22:07.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.write_buffer_manager: 0x55f2d1e501e0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.unordered_write: 0 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:22:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.row_cache: None 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.wal_filter: None 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.wal_compression: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:22:07.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:22:07.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_open_files: -1 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Compression algorithms supported: 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kZSTD supported: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:22:07.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.merge_operator: 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_filter: None 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:22:07.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f2d1e7dba0) 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: index_type: 0 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: data_block_index_type: 0 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: index_shortening: 1 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: checksum: 4 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: no_block_cache: 0 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_cache: 0x55f2d1e81090 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_cache_options: 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: capacity : 536870912 2024-04-25T00:22:07.900 INFO:journalctl@ceph.mon.b.smithi040.stdout: num_shard_bits : 4 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: strict_capacity_limit : 0 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_cache_compressed: (nil) 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: persistent_cache: (nil) 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_size: 4096 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_size_deviation: 10 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_restart_interval: 16 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: index_block_restart_interval: 1 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: metadata_block_size: 4096 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: partition_filters: 0 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: use_delta_encoding: 1 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: filter_policy: bloomfilter 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: whole_key_filtering: 1 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: verify_compression: 0 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:22:07.901 INFO:journalctl@ceph.mon.b.smithi040.stdout: format_version: 5 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout: enable_index_compression: 1 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout: block_align: 0 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout: max_auto_readahead_size: 262144 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout: prepopulate_block_cache: 0 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression: NoCompression 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.num_levels: 7 2024-04-25T00:22:07.902 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:22:07.903 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:22:07.904 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:22:07.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:22:07.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.ttl: 2592000 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:22:07.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 798c10c7-ef58-4290-98fa-fdc8df41f45f 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714004527869490, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714004527870118, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714004527, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "798c10c7-ef58-4290-98fa-fdc8df41f45f", "db_session_id": "TODYHMTQ1MNGU3HF9YI5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714004527870231, "job": 1, "event": "recovery_finished"} 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f2d1f72000 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: DB pointer 0x55f2d1f5c000 2024-04-25T00:22:07.908 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: ** DB Stats ** 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:22:07.909 INFO:journalctl@ceph.mon.b.smithi040.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:22:07.910 INFO:journalctl@ceph.mon.b.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: Block cache BinnedLRUCache@0x55f2d1e81090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: using public_addr v2:172.21.15.40:0/0 -> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: starting mon.b rank -1 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(???) e0 preinit fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: e1 2024-04-25T00:22:07.911 INFO:journalctl@ceph.mon.b.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout: legacy client fscid: -1 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout: 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout: No filesystems configured 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:22:07.912 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Deploying cephadm binary to smithi169 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.913 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Added host smithi169 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2606705163' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2606705163' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:22:07.914 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.915 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c;count:3 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Deploying daemon mon.c on smithi169 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/2593487859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:22:07.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.a calling monitor election 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.c calling monitor election 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: monmap epoch 2 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: last_changed 2024-04-25T00:21:58.550773+0000 2024-04-25T00:22:07.917 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: min_mon_release 19 (squid) 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: election_strategy: 1 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: fsmap 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mgrmap e13: x(active, since 50s) 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: overall HEALTH_OK 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.918 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: Deploying daemon mon.b on smithi040 2024-04-25T00:22:07.919 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/2708331445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:07.919 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:07.919 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:07.919 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 ceph-mon[32945]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:22:08.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 bash[32918]: b28e66123773c25c33fff9905cb17391123d4860c0cd80972342982414b63d13 2024-04-25T00:22:08.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:07 smithi040 systemd[1]: Started Ceph mon.b for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:22:09.437 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:22:09.438 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:22:09.680 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: mon.a calling monitor election 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: mon.c calling monitor election 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: mon.b calling monitor election 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: monmap epoch 3 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: last_changed 2024-04-25T00:22:07.895604+0000 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: min_mon_release 19 (squid) 2024-04-25T00:22:13.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: election_strategy: 1 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: fsmap 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: mgrmap e13: x(active, since 59s) 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: overall HEALTH_OK 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:22:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: mon.a calling monitor election 2024-04-25T00:22:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: mon.c calling monitor election 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: mon.b calling monitor election 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:22:13.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: monmap epoch 3 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: last_changed 2024-04-25T00:22:07.895604+0000 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: min_mon_release 19 (squid) 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: election_strategy: 1 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: fsmap 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: mgrmap e13: x(active, since 59s) 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: overall HEALTH_OK 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.392 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.393 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.393 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:22:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: mon.a calling monitor election 2024-04-25T00:22:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: mon.c calling monitor election 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: mon.b calling monitor election 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:22:13.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: monmap epoch 3 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: last_changed 2024-04-25T00:22:07.895604+0000 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: created 2024-04-25T00:20:09.931607+0000 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: min_mon_release 19 (squid) 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: election_strategy: 1 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: fsmap 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: mgrmap e13: x(active, since 59s) 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: overall HEALTH_OK 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.398 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:13.398 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:14.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 -- 172.21.15.169:0/3360513166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02440035c0 msgr2=0x7f0244005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:14.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 --2- 172.21.15.169:0/3360513166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02440035c0 0x7f0244005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f025c101c80 tx=0x7f024c00feb0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 -- 172.21.15.169:0/3360513166 shutdown_connections 2024-04-25T00:22:14.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 --2- 172.21.15.169:0/3360513166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02440035c0 0x7f0244005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 -- 172.21.15.169:0/3360513166 >> 172.21.15.169:0/3360513166 conn(0x7f025c0fd840 msgr2=0x7f025c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:14.127 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 -- 172.21.15.169:0/3360513166 shutdown_connections 2024-04-25T00:22:14.127 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.125+0000 7f0260a34700 1 -- 172.21.15.169:0/3360513166 wait complete. 2024-04-25T00:22:14.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.126+0000 7f0260a34700 1 Processor -- start 2024-04-25T00:22:14.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.126+0000 7f0260a34700 1 -- start start 2024-04-25T00:22:14.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02440035c0 0x7f025c076120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:14.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 0x7f025c0749b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:14.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f025c074ef0 0x7f025c1aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:14.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f025c109c40 con 0x7f025c074ef0 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f025c109ac0 con 0x7f02440035c0 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f025c109dc0 con 0x7f025c074530 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 0x7f025c0749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 0x7f025c0749b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36844/0 (socket says 172.21.15.169:36844) 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 -- 172.21.15.169:0/3154261171 learned_addr learned my addr 172.21.15.169:0/3154261171 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f025ad9d700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f025c074ef0 0x7f025c1aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f025a59c700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02440035c0 0x7f025c076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 -- 172.21.15.169:0/3154261171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02440035c0 msgr2=0x7f025c076120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02440035c0 0x7f025c076120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 -- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f025c074ef0 msgr2=0x7f025c1aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f025c074ef0 0x7f025c1aa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 -- 172.21.15.169:0/3154261171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c1aaa00 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f025ad9d700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f025c074ef0 0x7f025c1aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0259d9b700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 0x7f025c0749b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f025000b780 tx=0x7f025000bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250004230 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02500043d0 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c1aacf0 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.127+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250011b80 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.128+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1ab1d0 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.128+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f025001a400 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.128+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c103490 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.128+0000 7f024b7fe700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f023c039580 0x7f023c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.129+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f025004be50 con 0x7f025c074530 2024-04-25T00:22:14.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.129+0000 7f025a59c700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f023c039580 0x7f023c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:14.132 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.129+0000 7f025a59c700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f023c039580 0x7f023c03ba40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f024c001eb0 tx=0x7f024c005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:14.134 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.134+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f025001a840 con 0x7f025c074530 2024-04-25T00:22:14.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:14.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:14.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:14.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:14.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:14.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:14.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:14.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:14.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:22:14.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.417+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f025c02db30 con 0x7f025c074530 2024-04-25T00:22:14.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.418+0000 7f024b7fe700 1 -- 172.21.15.169:0/3154261171 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f025004d030 con 0x7f025c074530 2024-04-25T00:22:14.419 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T00:22:14.419 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","modified":"2024-04-25T00:22:07.895604Z","created":"2024-04-25T00:20:09.931607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T00:22:14.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f023c039580 msgr2=0x7f023c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:14.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f023c039580 0x7f023c03ba40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f024c001eb0 tx=0x7f024c005e60 comp rx=0 tx=0).stop 2024-04-25T00:22:14.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 msgr2=0x7f025c0749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:14.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 0x7f025c0749b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f025000b780 tx=0x7f025000bb40 comp rx=0 tx=0).stop 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 shutdown_connections 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f023c039580 0x7f023c03ba40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.420+0000 7f0260a34700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f025c074ef0 0x7f025c1aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.421+0000 7f0260a34700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02440035c0 0x7f025c076120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.421+0000 7f0260a34700 1 --2- 172.21.15.169:0/3154261171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c074530 0x7f025c0749b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.421+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 >> 172.21.15.169:0/3154261171 conn(0x7f025c0fd840 msgr2=0x7f025c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.421+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 shutdown_connections 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:14.421+0000 7f0260a34700 1 -- 172.21.15.169:0/3154261171 wait complete. 2024-04-25T00:22:14.422 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 3 2024-04-25T00:22:15.084 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:15.085 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:14 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/3154261171' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:15.112 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T00:22:15.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T00:22:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:14 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/3154261171' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:15.380 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:14 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/3154261171' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:22:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T00:22:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T00:22:16.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Updating smithi031:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:22:16.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:16.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:16.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:16.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Updating smithi031:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:16.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Updating smithi031:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Updating smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Updating smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:16.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:17.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 -- 172.21.15.31:0/714153513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6220102e00 msgr2=0x7f62201031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:17.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 --2- 172.21.15.31:0/714153513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6220102e00 0x7f62201031e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6210009910 tx=0x7f6210009c20 comp rx=0 tx=0).stop 2024-04-25T00:22:17.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 -- 172.21.15.31:0/714153513 shutdown_connections 2024-04-25T00:22:17.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 --2- 172.21.15.31:0/714153513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6220102e00 0x7f62201031e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 -- 172.21.15.31:0/714153513 >> 172.21.15.31:0/714153513 conn(0x7f62200fc630 msgr2=0x7f62200fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 -- 172.21.15.31:0/714153513 shutdown_connections 2024-04-25T00:22:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.015+0000 7f6225eee700 1 -- 172.21.15.31:0/714153513 wait complete. 2024-04-25T00:22:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 Processor -- start 2024-04-25T00:22:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 -- start start 2024-04-25T00:22:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6220102e00 0x7f6220100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6220100fa0 0x7f6220101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 0x7f62201a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f622010c180 con 0x7f62201a3ba0 2024-04-25T00:22:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f622010c000 con 0x7f6220102e00 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f6225eee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f622010c300 con 0x7f6220100fa0 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f621ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 0x7f62201a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f621ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 0x7f62201a5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42434/0 (socket says 172.21.15.31:42434) 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.016+0000 7f621ffff700 1 -- 172.21.15.31:0/996884811 learned_addr learned my addr 172.21.15.31:0/996884811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621f7fe700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6220102e00 0x7f6220100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621effd700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6220100fa0 0x7f6220101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621ffff700 1 -- 172.21.15.31:0/996884811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6220100fa0 msgr2=0x7f6220101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621ffff700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6220100fa0 0x7f6220101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621ffff700 1 -- 172.21.15.31:0/996884811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6220102e00 msgr2=0x7f6220100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621ffff700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6220102e00 0x7f6220100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621ffff700 1 -- 172.21.15.31:0/996884811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62201a66c0 con 0x7f62201a3ba0 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621effd700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6220100fa0 0x7f6220101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621ffff700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 0x7f62201a5f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f621800ba50 tx=0x7f621800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218019070 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6218014440 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621800ce60 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62201a69b0 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.017+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62201a6ef0 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.018+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f621801c070 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.018+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6220063890 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.018+0000 7f621cff9700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6208039580 0x7f620803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.019+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f621804b870 con 0x7f62201a3ba0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.019+0000 7f621f7fe700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6208039580 0x7f620803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.019+0000 7f621f7fe700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6208039580 0x7f620803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f621001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:17.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.024+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6218012500 con 0x7f62201a3ba0 2024-04-25T00:22:17.196 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:17 smithi031 ceph-mon[29167]: Reconfiguring daemon mon.a on smithi031 2024-04-25T00:22:17.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:17 smithi031 ceph-mon[29167]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:17.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.237+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f62201a70c0 con 0x7f62201a3ba0 2024-04-25T00:22:17.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.237+0000 7f621cff9700 1 -- 172.21.15.31:0/996884811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f6218012760 con 0x7f62201a3ba0 2024-04-25T00:22:17.238 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:17.238 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-04-25T00:22:17.238 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:22:17.238 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-25T00:22:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6208039580 msgr2=0x7f620803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6208039580 0x7f620803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f621001a040 comp rx=0 tx=0).stop 2024-04-25T00:22:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 msgr2=0x7f62201a5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 0x7f62201a5f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f621800ba50 tx=0x7f621800bd60 comp rx=0 tx=0).stop 2024-04-25T00:22:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 shutdown_connections 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6208039580 0x7f620803ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62201a3ba0 0x7f62201a5f90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6220102e00 0x7f6220100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 --2- 172.21.15.31:0/996884811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6220100fa0 0x7f6220101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.239+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 >> 172.21.15.31:0/996884811 conn(0x7f62200fc630 msgr2=0x7f6220105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.240+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 shutdown_connections 2024-04-25T00:22:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:17.240+0000 7f6225eee700 1 -- 172.21.15.31:0/996884811 wait complete. 2024-04-25T00:22:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:17 smithi169 ceph-mon[33112]: Reconfiguring daemon mon.a on smithi031 2024-04-25T00:22:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:17 smithi169 ceph-mon[33112]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:17.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:17 smithi040 ceph-mon[32945]: Reconfiguring daemon mon.a on smithi031 2024-04-25T00:22:17.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:17 smithi040 ceph-mon[32945]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/996884811' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:18.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:18 smithi031 ceph-mon[29167]: Reconfiguring daemon mon.b on smithi040 2024-04-25T00:22:18.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/996884811' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:18.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:18 smithi169 ceph-mon[33112]: Reconfiguring daemon mon.b on smithi040 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/996884811' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:18.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:18 smithi040 ceph-mon[32945]: Reconfiguring daemon mon.b on smithi040 2024-04-25T00:22:18.399 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T00:22:18.472 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:22:18.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:22:18.501 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:22:18.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:18.566 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:22:18.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:22:18.594 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:22:18.595 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:18.659 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:22:18.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:22:18.690 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:22:18.690 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:18.759 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-04-25T00:22:18.759 INFO:tasks.cephadm:Adding mgr.y on smithi040 2024-04-25T00:22:18.759 INFO:tasks.cephadm:Adding mgr.z on smithi169 2024-04-25T00:22:18.760 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi031=x;smithi040=y;smithi169=z' 2024-04-25T00:22:19.039 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 -- 172.21.15.169:0/3817199563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720100c90 msgr2=0x7ff7201010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 --2- 172.21.15.169:0/3817199563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720100c90 0x7ff7201010b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff710009910 tx=0x7ff710009c20 comp rx=0 tx=0).stop 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 -- 172.21.15.169:0/3817199563 shutdown_connections 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 --2- 172.21.15.169:0/3817199563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720100c90 0x7ff7201010b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 -- 172.21.15.169:0/3817199563 >> 172.21.15.169:0/3817199563 conn(0x7ff7200fc230 msgr2=0x7ff7200fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 -- 172.21.15.169:0/3817199563 shutdown_connections 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.537+0000 7ff7250b1700 1 -- 172.21.15.169:0/3817199563 wait complete. 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 Processor -- start 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 -- start start 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 0x7ff720197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720197a40 0x7ff72019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff720197ec0 0x7ff72019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff720106bd0 con 0x7ff720197a40 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff720106a50 con 0x7ff720197ec0 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff7250b1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff720106d50 con 0x7ff720100c90 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff71ed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 0x7ff720197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff71ed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 0x7ff720197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36856/0 (socket says 172.21.15.169:36856) 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff71ed9d700 1 -- 172.21.15.169:0/169827786 learned_addr learned my addr 172.21.15.169:0/169827786 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff71e59c700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720197a40 0x7ff72019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.538+0000 7ff71f59e700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff720197ec0 0x7ff72019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff71ed9d700 1 -- 172.21.15.169:0/169827786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff720197ec0 msgr2=0x7ff72019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff71ed9d700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff720197ec0 0x7ff72019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff71ed9d700 1 -- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720197a40 msgr2=0x7ff72019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff71ed9d700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720197a40 0x7ff72019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff71ed9d700 1 -- 172.21.15.169:0/169827786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72019f910 con 0x7ff720100c90 2024-04-25T00:22:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff71ed9d700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 0x7ff720197500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff710005e00 tx=0x7ff710005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710027070 con 0x7ff720100c90 2024-04-25T00:22:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff71000be10 con 0x7ff720100c90 2024-04-25T00:22:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72019fba0 con 0x7ff720100c90 2024-04-25T00:22:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710022da0 con 0x7ff720100c90 2024-04-25T00:22:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.539+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a00d0 con 0x7ff720100c90 2024-04-25T00:22:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.540+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff71002a070 con 0x7ff720100c90 2024-04-25T00:22:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.540+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff720063890 con 0x7ff720100c90 2024-04-25T00:22:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.540+0000 7ff707fff700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff708039580 0x7ff70803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.540+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff71004d630 con 0x7ff720100c90 2024-04-25T00:22:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.540+0000 7ff71e59c700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff708039580 0x7ff70803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.541+0000 7ff71e59c700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff708039580 0x7ff70803ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff714009ad0 tx=0x7ff714009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:20.546 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.545+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff710013070 con 0x7ff720100c90 2024-04-25T00:22:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.760+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}) -- 0x7ff720062700 con 0x7ff708039580 2024-04-25T00:22:20.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.766+0000 7ff707fff700 1 -- 172.21.15.169:0/169827786 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff720062700 con 0x7ff708039580 2024-04-25T00:22:20.767 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mgr update... 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff708039580 msgr2=0x7ff70803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff708039580 0x7ff70803ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff714009ad0 tx=0x7ff714009210 comp rx=0 tx=0).stop 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 msgr2=0x7ff720197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 0x7ff720197500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff710005e00 tx=0x7ff710005020 comp rx=0 tx=0).stop 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 shutdown_connections 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff720197ec0 0x7ff72019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720100c90 0x7ff720197500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff708039580 0x7ff70803ba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 --2- 172.21.15.169:0/169827786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff720197a40 0x7ff72019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 >> 172.21.15.169:0/169827786 conn(0x7ff7200fc230 msgr2=0x7ff72018e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.768+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 shutdown_connections 2024-04-25T00:22:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:22:20.769+0000 7ff7250b1700 1 -- 172.21.15.169:0/169827786 wait complete. 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:20.897 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:20 smithi169 ceph-mon[33112]: Reconfiguring daemon mon.c on smithi169 2024-04-25T00:22:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:22:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:21.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:21.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:20 smithi031 ceph-mon[29167]: Reconfiguring daemon mon.c on smithi169 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:20 smithi040 ceph-mon[32945]: Reconfiguring daemon mon.c on smithi169 2024-04-25T00:22:22.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:21 smithi169 ceph-mon[33112]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:22.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:21 smithi169 ceph-mon[33112]: Saving service mgr spec with placement smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T00:22:22.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:22.021 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:21 smithi031 ceph-mon[29167]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:21 smithi031 ceph-mon[29167]: Saving service mgr spec with placement smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T00:22:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:22.038 DEBUG:teuthology.orchestra.run.smithi040:mgr.y> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T00:22:22.041 DEBUG:teuthology.orchestra.run.smithi169:mgr.z> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T00:22:22.044 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T00:22:22.044 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:22:22.044 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:22:22.060 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:22:22.061 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-04-25T00:22:22.066 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:21 smithi040 ceph-mon[32945]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:22.066 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:21 smithi040 ceph-mon[32945]: Saving service mgr spec with placement smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T00:22:22.066 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.066 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.066 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:22.067 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 00:21:23.609696737 +0000 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 00:16:50.708149968 +0000 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 00:16:50.708149968 +0000 2024-04-25T00:22:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 00:16:50.708149968 +0000 2024-04-25T00:22:22.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:22:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T00:22:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T00:22:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000276417 s, 1.9 MB/s 2024-04-25T00:22:22.184 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:22:22.241 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-04-25T00:22:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:22:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:22.297 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T00:22:22.297 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:22.297 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:22.298 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 00:21:23.640696280 +0000 2024-04-25T00:22:22.298 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 00:16:51.069144390 +0000 2024-04-25T00:22:22.298 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 00:16:51.069144390 +0000 2024-04-25T00:22:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 00:16:51.069144390 +0000 2024-04-25T00:22:22.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:22:22.360 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T00:22:22.361 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T00:22:22.361 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000204755 s, 2.5 MB/s 2024-04-25T00:22:22.362 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:22:22.418 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-04-25T00:22:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:22:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:22.473 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T00:22:22.473 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:22.473 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:22.473 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 00:21:23.671695823 +0000 2024-04-25T00:22:22.474 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 00:16:51.414139059 +0000 2024-04-25T00:22:22.474 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 00:16:51.414139059 +0000 2024-04-25T00:22:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 00:16:51.414139059 +0000 2024-04-25T00:22:22.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:22:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T00:22:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T00:22:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000189336 s, 2.7 MB/s 2024-04-25T00:22:22.537 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:22:22.594 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-04-25T00:22:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:22:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T00:22:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 00:21:23.701695380 +0000 2024-04-25T00:22:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 00:16:51.758133743 +0000 2024-04-25T00:22:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 00:16:51.758133743 +0000 2024-04-25T00:22:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 00:16:51.758133743 +0000 2024-04-25T00:22:22.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:22:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T00:22:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T00:22:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000189137 s, 2.7 MB/s 2024-04-25T00:22:22.713 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:22:22.769 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:22:22.769 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:22:22.789 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:22:22.789 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 00:22:07.306686479 +0000 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:16:51.263664846 +0000 2024-04-25T00:22:22.849 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:16:51.263664846 +0000 2024-04-25T00:22:22.850 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:16:51.263664846 +0000 2024-04-25T00:22:22.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:22:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T00:22:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T00:22:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000369742 s, 1.4 MB/s 2024-04-25T00:22:22.918 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:22:22.980 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-25T00:22:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:23.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:23.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:22:23.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:23.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 00:22:07.307686467 +0000 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:16:51.612660366 +0000 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:16:51.612660366 +0000 2024-04-25T00:22:23.039 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:16:51.612660366 +0000 2024-04-25T00:22:23.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:22:23.080 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:23.080 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:23.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:23.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:23.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:23.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:22:23.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:23.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:23.104 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:23.104 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:23.104 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:23.104 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:23.104 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:23.105 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:22:23.105 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:23.105 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T00:22:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T00:22:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000232687 s, 2.2 MB/s 2024-04-25T00:22:23.108 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:22:23.168 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-25T00:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T00:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 00:22:07.307686467 +0000 2024-04-25T00:22:23.227 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:16:51.970655771 +0000 2024-04-25T00:22:23.227 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:16:51.970655771 +0000 2024-04-25T00:22:23.228 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:16:51.970655771 +0000 2024-04-25T00:22:23.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:22:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T00:22:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T00:22:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000218874 s, 2.3 MB/s 2024-04-25T00:22:23.293 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:22:23.352 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-25T00:22:23.409 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:22:23.409 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:23.409 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T00:22:23.409 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:23.409 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:23.410 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 00:22:07.307686467 +0000 2024-04-25T00:22:23.410 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:16:52.315651343 +0000 2024-04-25T00:22:23.410 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:16:52.315651343 +0000 2024-04-25T00:22:23.410 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:16:52.315651343 +0000 2024-04-25T00:22:23.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:22:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T00:22:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T00:22:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000228701 s, 2.2 MB/s 2024-04-25T00:22:23.476 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:22:23.534 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:22:23.534 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:22:23.550 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:22:23.550 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 00:21:57.252725240 +0000 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:16:48.037594180 +0000 2024-04-25T00:22:23.606 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:16:48.037594180 +0000 2024-04-25T00:22:23.607 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:16:48.037594180 +0000 2024-04-25T00:22:23.607 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:22:23.670 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T00:22:23.670 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T00:22:23.670 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000190325 s, 2.7 MB/s 2024-04-25T00:22:23.671 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:22:23.727 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-25T00:22:23.784 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:22:23.784 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:23.784 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T00:22:23.784 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:23.785 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:23.785 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 00:21:57.252725240 +0000 2024-04-25T00:22:23.785 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:16:48.386586900 +0000 2024-04-25T00:22:23.785 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:16:48.386586900 +0000 2024-04-25T00:22:23.785 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:16:48.386586900 +0000 2024-04-25T00:22:23.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:22:24.005 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:23 smithi169 ceph-mon[33112]: Deploying daemon mgr.z on smithi169 2024-04-25T00:22:24.006 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T00:22:24.006 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T00:22:24.007 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000190754 s, 2.7 MB/s 2024-04-25T00:22:24.008 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:22:24.023 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-25T00:22:24.086 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 00:22:23.956171577 +0000 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:16:48.718579975 +0000 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:16:48.718579975 +0000 2024-04-25T00:22:24.087 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:16:48.718579975 +0000 2024-04-25T00:22:24.087 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:22:24.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:23 smithi040 ceph-mon[32945]: Deploying daemon mgr.z on smithi169 2024-04-25T00:22:24.194 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T00:22:24.194 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T00:22:24.195 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00018454 s, 2.8 MB/s 2024-04-25T00:22:24.195 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:22:24.211 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-25T00:22:24.267 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 00:22:24.146167637 +0000 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:16:49.066572716 +0000 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:16:49.066572716 +0000 2024-04-25T00:22:24.268 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:16:49.066572716 +0000 2024-04-25T00:22:24.268 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:22:24.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:23 smithi031 ceph-mon[29167]: Deploying daemon mgr.z on smithi169 2024-04-25T00:22:24.329 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 systemd[1]: Starting Ceph mgr.z for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:22:24.332 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T00:22:24.332 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T00:22:24.332 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000206165 s, 2.5 MB/s 2024-04-25T00:22:24.334 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:22:24.390 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-04-25T00:22:24.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:22:24.639 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 podman[35079]: 2024-04-25 00:22:24.42040223 +0000 UTC m=+0.103458739 container create cfba88e51db82ae8ae5c5bf9cdc67e8327267d9d12bdcbefb72ebaa3b1d82a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T00:22:24.639 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 podman[35079]: 2024-04-25 00:22:24.336794377 +0000 UTC m=+0.019850902 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:22:24.639 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 podman[35079]: 2024-04-25 00:22:24.639482711 +0000 UTC m=+0.322539225 container init cfba88e51db82ae8ae5c5bf9cdc67e8327267d9d12bdcbefb72ebaa3b1d82a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:22:24.644 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:22:24.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:24 smithi169 ceph-mon[33112]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:24.890 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 podman[35079]: 2024-04-25 00:22:24.642070275 +0000 UTC m=+0.325126793 container start cfba88e51db82ae8ae5c5bf9cdc67e8327267d9d12bdcbefb72ebaa3b1d82a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T00:22:24.890 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 ceph-mgr[35134]: -- 172.21.15.169:0/1111748688 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e844a8a5a0 con 0x55e844a5e800 2024-04-25T00:22:24.890 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 bash[35079]: cfba88e51db82ae8ae5c5bf9cdc67e8327267d9d12bdcbefb72ebaa3b1d82a66 2024-04-25T00:22:24.890 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 systemd[1]: Started Ceph mgr.z for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:22:24.890 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:24.791+0000 7f0de13b1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:22:25.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:24 smithi031 ceph-mon[29167]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:24 smithi040 ceph-mon[32945]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:25.388 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:24.979+0000 7f0de13b1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:22:25.388 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:25.081+0000 7f0de13b1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:22:25.388 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:25.387+0000 7f0de13b1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:22:25.639 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:25.472+0000 7f0de13b1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:22:25.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:25.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:25.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:25.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:25.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:22:25.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:25 smithi031 ceph-mon[29167]: Deploying daemon mgr.y on smithi040 2024-04-25T00:22:26.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:25 smithi169 ceph-mon[33112]: Deploying daemon mgr.y on smithi040 2024-04-25T00:22:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:22:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:26.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:26.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:25 smithi040 ceph-mon[32945]: Deploying daemon mgr.y on smithi040 2024-04-25T00:22:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:22:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:22:26.508 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T00:22:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T00:22:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129914 s, 807 MB/s 2024-04-25T00:22:26.508 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T00:22:27.006 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:26 smithi169 ceph-mon[33112]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:27.006 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:26.665+0000 7f0de13b1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:22:27.006 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:26.833+0000 7f0de13b1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:22:27.006 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:27.005+0000 7f0de13b1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:22:27.092 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:26 smithi031 ceph-mon[29167]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:27.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-04-25T00:22:27.129 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:26 smithi040 ceph-mon[32945]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:27.129 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:26 smithi040 systemd[1]: Starting Ceph mgr.y for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:22:27.372 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:22:27.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 podman[34324]: 2024-04-25 00:22:27.129306369 +0000 UTC m=+0.098695077 container create 889156ed9c113922f9b0960f181a4791dbbc55b5ab4b1f0b8f29dc303b4b013b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:22:27.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 podman[34324]: 2024-04-25 00:22:27.048658466 +0000 UTC m=+0.018047176 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:22:27.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 podman[34324]: 2024-04-25 00:22:27.323219872 +0000 UTC m=+0.292608583 container init 889156ed9c113922f9b0960f181a4791dbbc55b5ab4b1f0b8f29dc303b4b013b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T00:22:27.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 podman[34324]: 2024-04-25 00:22:27.326572343 +0000 UTC m=+0.295961051 container start 889156ed9c113922f9b0960f181a4791dbbc55b5ab4b1f0b8f29dc303b4b013b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T00:22:27.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 ceph-mgr[34343]: -- 172.21.15.40:0/131033279 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649e91d65a0 con 0x5649e91aac00 2024-04-25T00:22:27.584 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:27.289+0000 7f0de13b1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:22:27.584 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:27.380+0000 7f0de13b1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:22:27.584 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:27.583+0000 7f0de13b1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:22:27.678 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 bash[34324]: 889156ed9c113922f9b0960f181a4791dbbc55b5ab4b1f0b8f29dc303b4b013b 2024-04-25T00:22:27.678 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 systemd[1]: Started Ceph mgr.y for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:22:27.678 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:27.486+0000 7fe0ef615200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:22:27.678 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:27.677+0000 7fe0ef615200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:22:27.889 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:27.778+0000 7f0de13b1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:22:28.115 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:27 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:27.785+0000 7fe0ef615200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:22:28.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:28.115+0000 7fe0ef615200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:22:28.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:28.209+0000 7fe0ef615200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:28.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.751 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:28.425+0000 7f0de13b1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:28.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:28.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:28.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:28.899 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:29.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 -- 172.21.15.31:0/2252530295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec102d10 msgr2=0x7f73ec105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:29.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/2252530295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec102d10 0x7f73ec105100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009990 tx=0x7f73e8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:22:29.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 -- 172.21.15.31:0/2252530295 shutdown_connections 2024-04-25T00:22:29.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/2252530295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec102d10 0x7f73ec105100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:29.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/2252530295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73ec107f70 0x7f73ec10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/2252530295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 0x7f73ec107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 -- 172.21.15.31:0/2252530295 >> 172.21.15.31:0/2252530295 conn(0x7f73ec0fc720 msgr2=0x7f73ec0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 -- 172.21.15.31:0/2252530295 shutdown_connections 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 -- 172.21.15.31:0/2252530295 wait complete. 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.087+0000 7f73f3cfc700 1 Processor -- start 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 -- start start 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73ec102d10 0x7f73ec198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 0x7f73ec198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 0x7f73ec19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:29.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10dd40 con 0x7f73ec107f70 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10dbc0 con 0x7f73ec102d10 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f3cfc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10dec0 con 0x7f73ec105640 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 0x7f73ec19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f1297700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 0x7f73ec198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 0x7f73ec19f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55850/0 (socket says 172.21.15.31:55850) 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 -- 172.21.15.31:0/1661972633 learned_addr learned my addr 172.21.15.31:0/1661972633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:22:29.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f1a98700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73ec102d10 0x7f73ec198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 -- 172.21.15.31:0/1661972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 msgr2=0x7f73ec198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 0x7f73ec198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 -- 172.21.15.31:0/1661972633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73ec102d10 msgr2=0x7f73ec198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73ec102d10 0x7f73ec198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 -- 172.21.15.31:0/1661972633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec1a0070 con 0x7f73ec107f70 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f1297700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 0x7f73ec198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.088+0000 7f73f2299700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 0x7f73ec19f9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007ef0 tx=0x7f73e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.089+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4004bb0 con 0x7f73ec107f70 2024-04-25T00:22:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.089+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec100b60 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.089+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e4004d50 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.089+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4004400 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.089+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1010a0 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.090+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f73e401a030 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.090+0000 7f73e2ffd700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73d8039330 0x7f73d803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.090+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f73e404b930 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.090+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec068f20 con 0x7f73ec107f70 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.090+0000 7f73f1a98700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73d8039330 0x7f73d803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:29.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.090+0000 7f73f1a98700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73d8039330 0x7f73d803b7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b710 tx=0x7f73e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:29.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.095+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73e4021020 con 0x7f73ec107f70 2024-04-25T00:22:29.139 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:28.857+0000 7f0de13b1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:22:29.139 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:28.954+0000 7f0de13b1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:22:29.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:29.314+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f73ec02df10 con 0x7f73d8039330 2024-04-25T00:22:29.639 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:29 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:29.162+0000 7f0de13b1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:22:29.871 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:29 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:29.522+0000 7fe0ef615200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:22:29.871 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:29 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:29.696+0000 7fe0ef615200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:22:29.871 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:29 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:29.871+0000 7fe0ef615200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:22:30.018 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:29 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:29.700+0000 7f0de13b1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:22:30.018 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:29 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:29.791+0000 7f0de13b1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:30.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:30.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:30.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:22:30.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:22:30.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:30 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:30.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:30.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:22:30.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:22:30.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:30.030+0000 7f0de13b1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:22:30.374 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:30.164+0000 7f0de13b1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:22:30.375 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:30.258+0000 7f0de13b1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:22:30.375 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:30.372+0000 7f0de13b1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:22:30.470 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:30.167+0000 7fe0ef615200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:22:30.470 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:30.262+0000 7fe0ef615200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:22:30.470 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:30.470+0000 7fe0ef615200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:22:30.639 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:30.462+0000 7f0de13b1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:22:30.895 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:30.667+0000 7fe0ef615200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:22:31.389 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:31.275+0000 7f0de13b1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:31.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.394 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:31.289+0000 7fe0ef615200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:31.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:31.889 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:22:31 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z[35130]: 2024-04-25T00:22:31.488+0000 7f0de13b1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:22:32.029 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:31.725+0000 7fe0ef615200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:22:32.030 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:31.823+0000 7fe0ef615200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:22:32.030 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:32.029+0000 7fe0ef615200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:22:32.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:32.085+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f73e401ca20 con 0x7f73ec107f70 2024-04-25T00:22:32.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:32 smithi031 ceph-mon[29167]: Standby manager daemon z started 2024-04-25T00:22:32.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:32 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:22:32.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:32 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:22:32.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:32 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:22:32.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:32 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:22:32.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:32 smithi169 ceph-mon[33112]: Standby manager daemon z started 2024-04-25T00:22:32.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:32 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:22:32.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:32 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:22:32.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:32 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:22:32.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:32 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:22:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-mon[32945]: Standby manager daemon z started 2024-04-25T00:22:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:22:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:22:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:22:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.169:0/3952529881' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:22:32.895 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:32.569+0000 7fe0ef615200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:22:32.895 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:32.663+0000 7fe0ef615200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:22:33.266 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:33.266 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T00:22:33.266 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:22:33.266 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2793266219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5605681-c8ce-4528-a557-32de873bf2a3"}]: dispatch 2024-04-25T00:22:33.266 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2793266219' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5605681-c8ce-4528-a557-32de873bf2a3"}]': finished 2024-04-25T00:22:33.266 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:22:33.267 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:33.267 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:32.911+0000 7fe0ef615200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:22:33.267 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:33.049+0000 7fe0ef615200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:22:33.267 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:33.147+0000 7fe0ef615200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:22:33.267 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:33.266+0000 7fe0ef615200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2793266219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5605681-c8ce-4528-a557-32de873bf2a3"}]: dispatch 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2793266219' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5605681-c8ce-4528-a557-32de873bf2a3"}]': finished 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:22:33.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:33.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:33.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T00:22:33.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:22:33.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2793266219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5605681-c8ce-4528-a557-32de873bf2a3"}]: dispatch 2024-04-25T00:22:33.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2793266219' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5605681-c8ce-4528-a557-32de873bf2a3"}]': finished 2024-04-25T00:22:33.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:22:33.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:33.645 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:33.359+0000 7fe0ef615200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:22:34.347 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2157730568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:22:34.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2157730568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:22:34.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2157730568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:22:34.395 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:34 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:34.198+0000 7fe0ef615200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:22:34.879 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:22:34 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y[34339]: 2024-04-25T00:22:34.417+0000 7fe0ef615200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:22:35.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:35.100+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f73e4017d30 con 0x7f73ec107f70 2024-04-25T00:22:35.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:35 smithi040 ceph-mon[32945]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:35.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:35 smithi040 ceph-mon[32945]: Standby manager daemon y started 2024-04-25T00:22:35.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:35 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:22:35.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:35 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:22:35.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:35 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:22:35.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:35 smithi040 ceph-mon[32945]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:22:35.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:35 smithi169 ceph-mon[33112]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:35.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:35 smithi169 ceph-mon[33112]: Standby manager daemon y started 2024-04-25T00:22:35.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:35 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:22:35.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:35 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:22:35.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:35 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:22:35.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:35 smithi169 ceph-mon[33112]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:22:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:35 smithi031 ceph-mon[29167]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:35 smithi031 ceph-mon[29167]: Standby manager daemon y started 2024-04-25T00:22:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:35 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:22:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:35 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:22:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:35 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:22:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:35 smithi031 ceph-mon[29167]: from='mgr.? 172.21.15.40:0/4005668154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:22:36.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:36 smithi169 ceph-mon[33112]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T00:22:36.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:36 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:22:36.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:36 smithi040 ceph-mon[32945]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T00:22:36.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:36 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:22:36.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:36 smithi031 ceph-mon[29167]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T00:22:36.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:22:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:37 smithi031 ceph-mon[29167]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:37 smithi169 ceph-mon[33112]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:37.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:37 smithi040 ceph-mon[32945]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:39 smithi169 ceph-mon[33112]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:39 smithi040 ceph-mon[32945]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:39.460 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:39 smithi031 ceph-mon[29167]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:41.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:41 smithi169 ceph-mon[33112]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:41.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:22:41.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:41 smithi040 ceph-mon[32945]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:22:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:41.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:41 smithi031 ceph-mon[29167]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:41.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:22:41.407 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:42.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:42 smithi169 ceph-mon[33112]: Deploying daemon osd.0 on smithi031 2024-04-25T00:22:42.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:42 smithi040 ceph-mon[32945]: Deploying daemon osd.0 on smithi031 2024-04-25T00:22:42.399 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:42 smithi031 ceph-mon[29167]: Deploying daemon osd.0 on smithi031 2024-04-25T00:22:43.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:43 smithi169 ceph-mon[33112]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:43 smithi040 ceph-mon[32945]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:43 smithi031 ceph-mon[29167]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:44.245 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:44 smithi031 ceph-mon[29167]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:44.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:44 smithi169 ceph-mon[33112]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:44 smithi040 ceph-mon[32945]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:46 smithi031 ceph-mon[29167]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:47.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:46 smithi169 ceph-mon[33112]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:46 smithi040 ceph-mon[32945]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:49.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:48 smithi031 ceph-mon[29167]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:49.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:49.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:49.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:48 smithi169 ceph-mon[33112]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:48 smithi040 ceph-mon[32945]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:49.476 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-04-25T00:22:49.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.475+0000 7f73e2ffd700 1 -- 172.21.15.31:0/1661972633 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f73ec02df10 con 0x7f73d8039330 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73d8039330 msgr2=0x7f73d803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73d8039330 0x7f73d803b7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b710 tx=0x7f73e8005fd0 comp rx=0 tx=0).stop 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 msgr2=0x7f73ec19f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 0x7f73ec19f9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007ef0 tx=0x7f73e400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 shutdown_connections 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73d8039330 0x7f73d803b7f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec107f70 0x7f73ec19f9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73ec102d10 0x7f73ec198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 --2- 172.21.15.31:0/1661972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73ec105640 0x7f73ec198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 >> 172.21.15.31:0/1661972633 conn(0x7f73ec0fc720 msgr2=0x7f73ec103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 shutdown_connections 2024-04-25T00:22:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:22:49.478+0000 7f73f3cfc700 1 -- 172.21.15.31:0/1661972633 wait complete. 2024-04-25T00:22:50.148 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T00:22:50.151 INFO:tasks.cephadm:Deploying osd.1 on smithi040 with /dev/vg_nvme/lv_4... 2024-04-25T00:22:50.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:22:50.403 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:22:50.524 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:50 smithi040 ceph-mon[32945]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:50.525 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:50.525 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:50.525 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:50.525 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:50 smithi040 ceph-mon[32945]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:22:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:50 smithi031 ceph-mon[29167]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:50 smithi031 ceph-mon[29167]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:22:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:50 smithi169 ceph-mon[33112]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:50 smithi169 ceph-mon[33112]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:22:51.548 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:51 smithi040 ceph-mon[32945]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:22:51.548 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:51 smithi040 ceph-mon[32945]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:22:51.549 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:51 smithi040 ceph-mon[32945]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:22:51.549 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:51 smithi031 ceph-mon[29167]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:22:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:51 smithi031 ceph-mon[29167]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:22:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:51 smithi031 ceph-mon[29167]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:22:51.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:51.772 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:22:51 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0[41409]: 2024-04-25T00:22:51.446+0000 7fe476966700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:22:51.773 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:22:51 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0[41409]: 2024-04-25T00:22:51.449+0000 7fe472759700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:22:51.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:51 smithi169 ceph-mon[33112]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:22:51.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:51 smithi169 ceph-mon[33112]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:22:51.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:51 smithi169 ceph-mon[33112]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:22:51.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:22:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:22:52.180 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T00:22:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T00:22:52.181 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109186 s, 960 MB/s 2024-04-25T00:22:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T00:22:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:52 smithi031 ceph-mon[29167]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:52 smithi031 ceph-mon[29167]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:22:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:52 smithi031 ceph-mon[29167]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:22:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:52.752 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:52 smithi040 ceph-mon[32945]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:52.752 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:52 smithi040 ceph-mon[32945]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:22:52.752 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:52 smithi040 ceph-mon[32945]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:22:52.752 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:52.753 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:52.780 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_4 2024-04-25T00:22:52.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:52 smithi169 ceph-mon[33112]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:22:52.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:52 smithi169 ceph-mon[33112]: from='osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:22:52.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:52 smithi169 ceph-mon[33112]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:22:52.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:52.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:22:53.682 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] boot 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi031 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi031 to 17093M 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:53.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:22:53.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:22:53.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] boot 2024-04-25T00:22:53.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:22:53.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:53.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi031 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi031 to 17093M 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:53.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: osd.0 [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] boot 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi031 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.792 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:53.792 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi031 to 17093M 2024-04-25T00:22:53.792 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:53.792 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:53.792 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:53.792 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 -- 172.21.15.40:0/2266199340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 msgr2=0x7f00ec1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 --2- 172.21.15.40:0/2266199340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec1095e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00e400b0d0 tx=0x7f00e400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 -- 172.21.15.40:0/2266199340 shutdown_connections 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 --2- 172.21.15.40:0/2266199340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec1095e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 --2- 172.21.15.40:0/2266199340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 0x7f00ec1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 --2- 172.21.15.40:0/2266199340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00ec102390 0x7f00ec1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 -- 172.21.15.40:0/2266199340 >> 172.21.15.40:0/2266199340 conn(0x7f00ec0fc730 msgr2=0x7f00ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 -- 172.21.15.40:0/2266199340 shutdown_connections 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.377+0000 7f00f2198700 1 -- 172.21.15.40:0/2266199340 wait complete. 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 Processor -- start 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 -- start start 2024-04-25T00:22:54.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 0x7f00ec06aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00ec102390 0x7f00ec19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec10cc30 con 0x7f00ec102390 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec10cab0 con 0x7f00ec102d30 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00f2198700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec10cdb0 con 0x7f00ec101190 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00eb7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 0x7f00ec06aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:54.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43634/0 (socket says 172.21.15.40:43634) 2024-04-25T00:22:54.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.378+0000 7f00ebfff700 1 -- 172.21.15.40:0/2292890925 learned_addr learned my addr 172.21.15.40:0/2292890925 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T00:22:54.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00eaffd700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00ec102390 0x7f00ec19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:54.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00ebfff700 1 -- 172.21.15.40:0/2292890925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 msgr2=0x7f00ec06aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:54.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00ebfff700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 0x7f00ec06aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:54.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00ebfff700 1 -- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00ec102390 msgr2=0x7f00ec19d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:54.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00ebfff700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00ec102390 0x7f00ec19d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00ebfff700 1 -- 172.21.15.40:0/2292890925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec1a0000 con 0x7f00ec102d30 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00eb7fe700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 0x7f00ec06aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00ebfff700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec19f730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00e400d010 tx=0x7f00e4007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4021070 con 0x7f00ec102d30 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00e4007e50 con 0x7f00ec102d30 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec1a0290 con 0x7f00ec102d30 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e401bb00 con 0x7f00ec102d30 2024-04-25T00:22:54.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.379+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec1a0740 con 0x7f00ec102d30 2024-04-25T00:22:54.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.380+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec068f20 con 0x7f00ec102d30 2024-04-25T00:22:54.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.381+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f00e4024070 con 0x7f00ec102d30 2024-04-25T00:22:54.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.382+0000 7f00e8ff9700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f00d40a2b90 0x7f00d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:54.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.382+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f00e400f020 con 0x7f00ec102d30 2024-04-25T00:22:54.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.382+0000 7f00eb7fe700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f00d40a2b90 0x7f00d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:54.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.383+0000 7f00eb7fe700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f00d40a2b90 0x7f00d40a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00ec1021f0 tx=0x7f00dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:54.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.387+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f00e402b020 con 0x7f00ec102d30 2024-04-25T00:22:54.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:22:54.608+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f00ec062670 con 0x7f00d40a2b90 2024-04-25T00:22:54.608 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:54 smithi040 ceph-mon[32945]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:54.608 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:54 smithi040 ceph-mon[32945]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:22:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:54 smithi031 ceph-mon[29167]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:54 smithi031 ceph-mon[29167]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:22:54.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:54 smithi169 ceph-mon[33112]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:54.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:54 smithi169 ceph-mon[33112]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:22:55.564 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:55 smithi040 ceph-mon[32945]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:55.564 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:22:55.564 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:22:55.564 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:55 smithi031 ceph-mon[29167]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:22:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:22:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:55.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:55 smithi169 ceph-mon[33112]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:55.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:22:55.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:22:55.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:56.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:56 smithi040 ceph-mon[32945]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:56.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:56 smithi031 ceph-mon[29167]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:56.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:56 smithi169 ceph-mon[33112]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:57.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:57 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/3851153831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]: dispatch 2024-04-25T00:22:57.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:57 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]: dispatch 2024-04-25T00:22:57.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:57 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]': finished 2024-04-25T00:22:57.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:57 smithi031 ceph-mon[29167]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:22:57.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:22:57.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:57 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/3851153831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]: dispatch 2024-04-25T00:22:57.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:57 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]: dispatch 2024-04-25T00:22:57.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:57 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]': finished 2024-04-25T00:22:57.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:57 smithi169 ceph-mon[33112]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:22:57.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:22:57.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:57 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/3851153831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]: dispatch 2024-04-25T00:22:57.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:57 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]: dispatch 2024-04-25T00:22:57.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:57 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32f950d6-4574-449b-8273-82833bc7774d"}]': finished 2024-04-25T00:22:57.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:57 smithi040 ceph-mon[32945]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:22:57.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:22:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:58 smithi031 ceph-mon[29167]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:22:58 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/1796200547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:22:58.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:58 smithi169 ceph-mon[33112]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:58.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:22:58 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/1796200547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:22:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:58 smithi040 ceph-mon[32945]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:22:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:22:58 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/1796200547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:00.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:00 smithi031 ceph-mon[29167]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:00.848 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:00 smithi040 ceph-mon[32945]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:00 smithi169 ceph-mon[33112]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:02.512 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:02 smithi040 ceph-mon[32945]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:02.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:02 smithi031 ceph-mon[29167]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:02.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:02 smithi169 ceph-mon[33112]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:04.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:04 smithi040 ceph-mon[32945]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:04.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:23:04.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:04.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:04 smithi031 ceph-mon[29167]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:04.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:23:04.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:04 smithi169 ceph-mon[33112]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:23:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:05.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:05 smithi040 ceph-mon[32945]: Deploying daemon osd.1 on smithi040 2024-04-25T00:23:05.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:05 smithi031 ceph-mon[29167]: Deploying daemon osd.1 on smithi040 2024-04-25T00:23:05.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:05 smithi169 ceph-mon[33112]: Deploying daemon osd.1 on smithi040 2024-04-25T00:23:06.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:06 smithi031 ceph-mon[29167]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:06 smithi169 ceph-mon[33112]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:06 smithi040 ceph-mon[32945]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:08.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:08 smithi040 ceph-mon[32945]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:08.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:08 smithi031 ceph-mon[29167]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:08.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:08 smithi169 ceph-mon[33112]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:11.258 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:10 smithi040 ceph-mon[32945]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:11.259 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:11.259 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:11.259 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:10 smithi031 ceph-mon[29167]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:11.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:10 smithi169 ceph-mon[33112]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:11.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:11.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:11.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:12 smithi031 ceph-mon[29167]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:13.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.293 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:12 smithi040 ceph-mon[32945]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:13.293 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.293 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.293 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:13.293 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:13.294 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:12 smithi169 ceph-mon[33112]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:13.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.676+0000 7f00e8ff9700 1 -- 172.21.15.40:0/2292890925 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f00ec062670 con 0x7f00d40a2b90 2024-04-25T00:23:13.678 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 1 on host 'smithi040' 2024-04-25T00:23:13.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f00d40a2b90 msgr2=0x7f00d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:13.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f00d40a2b90 0x7f00d40a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00ec1021f0 tx=0x7f00dc00b410 comp rx=0 tx=0).stop 2024-04-25T00:23:13.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 msgr2=0x7f00ec19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:13.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec19f730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00e400d010 tx=0x7f00e4007a20 comp rx=0 tx=0).stop 2024-04-25T00:23:13.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 shutdown_connections 2024-04-25T00:23:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ec102d30 0x7f00ec19f730 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00ec101190 0x7f00ec06aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f00d40a2b90 0x7f00d40a5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 --2- 172.21.15.40:0/2292890925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00ec102390 0x7f00ec19d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 >> 172.21.15.40:0/2292890925 conn(0x7f00ec0fc730 msgr2=0x7f00ec105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.678+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 shutdown_connections 2024-04-25T00:23:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:23:13.679+0000 7f00f2198700 1 -- 172.21.15.40:0/2292890925 wait complete. 2024-04-25T00:23:14.316 DEBUG:teuthology.orchestra.run.smithi040:osd.1> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T00:23:14.318 INFO:tasks.cephadm:Deploying osd.2 on smithi169 with /dev/vg_nvme/lv_4... 2024-04-25T00:23:14.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:23:14.565 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:23:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:14 smithi169 ceph-mon[33112]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:14 smithi169 ceph-mon[33112]: from='osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:23:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:14 smithi169 ceph-mon[33112]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:23:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:14 smithi040 ceph-mon[32945]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:14 smithi040 ceph-mon[32945]: from='osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:23:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:14 smithi040 ceph-mon[32945]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:14 smithi031 ceph-mon[29167]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:14 smithi031 ceph-mon[29167]: from='osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:14 smithi031 ceph-mon[29167]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:23:15.775 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:15 smithi040 ceph-mon[32945]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:23:15.775 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:15 smithi040 ceph-mon[32945]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:23:15.776 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:15.776 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:15 smithi040 ceph-mon[32945]: from='osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:23:15.776 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:15 smithi040 ceph-mon[32945]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:23:15.830 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:15 smithi169 ceph-mon[33112]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:23:15.830 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:15 smithi169 ceph-mon[33112]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:23:15.830 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:15.830 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:15 smithi169 ceph-mon[33112]: from='osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:23:15.830 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:15 smithi169 ceph-mon[33112]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:23:15.910 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:15 smithi031 ceph-mon[29167]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:23:15.911 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:15 smithi031 ceph-mon[29167]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:23:15.911 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:15.911 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:15 smithi031 ceph-mon[29167]: from='osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:23:15.911 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:15 smithi031 ceph-mon[29167]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:23:16.390 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:23:16.390 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:23:16.390 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T00:23:16.391 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T00:23:16.391 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115859 s, 905 MB/s 2024-04-25T00:23:16.391 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T00:23:16.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1[37546]: 2024-04-25T00:23:16.140+0000 7f47108bd700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:23:16.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1[37546]: 2024-04-25T00:23:16.146+0000 7f47086a8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:23:16.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:16 smithi169 ceph-mon[33112]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:16.968 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:16 smithi169 ceph-mon[33112]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:23:16.968 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:16 smithi169 ceph-mon[33112]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:23:16.968 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:16.968 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:16.968 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:16 smithi169 ceph-mon[33112]: from='osd.1 ' entity='osd.1' 2024-04-25T00:23:16.986 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-mon[32945]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:16.986 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-mon[32945]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:23:16.986 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-mon[32945]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:23:16.986 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:16.986 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:16.986 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:16 smithi040 ceph-mon[32945]: from='osd.1 ' entity='osd.1' 2024-04-25T00:23:16.994 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-04-25T00:23:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:16 smithi031 ceph-mon[29167]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:16 smithi031 ceph-mon[29167]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:23:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:16 smithi031 ceph-mon[29167]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:23:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:16 smithi031 ceph-mon[29167]: from='osd.1 ' entity='osd.1' 2024-04-25T00:23:17.239 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:23:17.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] boot 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:17.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:17.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:17 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:23:18.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:23:18.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:18.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] boot 2024-04-25T00:23:18.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:18.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:17 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: osd.1 [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] boot 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:23:18.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:23:18.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:18.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:18.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:17 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:18.691 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 -- 172.21.15.169:0/107834219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300103ec0 msgr2=0x7f130010a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 --2- 172.21.15.169:0/107834219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300103ec0 0x7f130010a770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b0a0 tx=0x7f12f800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 -- 172.21.15.169:0/107834219 shutdown_connections 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 --2- 172.21.15.169:0/107834219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300103ec0 0x7f130010a770 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 --2- 172.21.15.169:0/107834219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1300100540 0x7f1300100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 --2- 172.21.15.169:0/107834219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300103520 0x7f1300103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 -- 172.21.15.169:0/107834219 >> 172.21.15.169:0/107834219 conn(0x7f13000fbec0 msgr2=0x7f13000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 -- 172.21.15.169:0/107834219 shutdown_connections 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.688+0000 7f1307879700 1 -- 172.21.15.169:0/107834219 wait complete. 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 Processor -- start 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 -- start start 2024-04-25T00:23:18.692 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 0x7f1300198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300103520 0x7f1300198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1300103ec0 0x7f130019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f130010dd00 con 0x7f1300103520 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f130010db80 con 0x7f1300103ec0 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1307879700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f130010de80 con 0x7f1300100540 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1305615700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 0x7f1300198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1305615700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 0x7f1300198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41248/0 (socket says 172.21.15.169:41248) 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1305615700 1 -- 172.21.15.169:0/2079355999 learned_addr learned my addr 172.21.15.169:0/2079355999 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1304e14700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300103520 0x7f1300198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.689+0000 7f1305e16700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1300103ec0 0x7f130019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1305615700 1 -- 172.21.15.169:0/2079355999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1300103ec0 msgr2=0x7f130019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:18.693 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1305615700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1300103ec0 0x7f130019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1305615700 1 -- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300103520 msgr2=0x7f1300198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1305615700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300103520 0x7f1300198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1305615700 1 -- 172.21.15.169:0/2079355999 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130019ff70 con 0x7f1300100540 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1305615700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 0x7f1300198310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00c920 tx=0x7f12fc00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc004220 con 0x7f1300100540 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13001a0260 con 0x7f1300100540 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12fc0043c0 con 0x7f1300100540 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc003bf0 con 0x7f1300100540 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.690+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13001a07a0 con 0x7f1300100540 2024-04-25T00:23:18.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.691+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1300068f20 con 0x7f1300100540 2024-04-25T00:23:18.696 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.692+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f12fc0076d0 con 0x7f1300100540 2024-04-25T00:23:18.696 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.693+0000 7f12f67fc700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ec0a2c10 0x7f12ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:18.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.693+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f12fc0cd220 con 0x7f1300100540 2024-04-25T00:23:18.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.693+0000 7f1304e14700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ec0a2c10 0x7f12ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:18.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.693+0000 7f1304e14700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ec0a2c10 0x7f12ec0a50d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005950 tx=0x7f12f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:18.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.698+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12fc014030 con 0x7f1300100540 2024-04-25T00:23:18.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:18.899+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1300062670 con 0x7f12ec0a2c10 2024-04-25T00:23:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:18 smithi031 ceph-mon[29167]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:18 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi040 2024-04-25T00:23:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:18 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi040 to 17093M 2024-04-25T00:23:19.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:18 smithi169 ceph-mon[33112]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:19.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:18 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi040 2024-04-25T00:23:19.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:18 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi040 to 17093M 2024-04-25T00:23:19.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:18 smithi040 ceph-mon[32945]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:23:19.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:18 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi040 2024-04-25T00:23:19.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:18 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi040 to 17093M 2024-04-25T00:23:19.788 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:19 smithi169 ceph-mon[33112]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:23:19.788 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:19 smithi169 ceph-mon[33112]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:23:19.788 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:19 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:23:19.788 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:19 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:23:19.788 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:19 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:20.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:19 smithi031 ceph-mon[29167]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:23:20.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:19 smithi031 ceph-mon[29167]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:23:20.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:19 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:23:20.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:19 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:23:20.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:19 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:20.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:19 smithi040 ceph-mon[32945]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:23:20.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:19 smithi040 ceph-mon[32945]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:23:20.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:19 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:23:20.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:19 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:23:20.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:19 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:20.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:20 smithi169 ceph-mon[33112]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:20 smithi031 ceph-mon[29167]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:21.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:20 smithi040 ceph-mon[32945]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/377022102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]: dispatch 2024-04-25T00:23:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:21 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]: dispatch 2024-04-25T00:23:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:21 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]': finished 2024-04-25T00:23:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:21 smithi031 ceph-mon[29167]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:23:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:22.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/377022102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]: dispatch 2024-04-25T00:23:22.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:21 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]: dispatch 2024-04-25T00:23:22.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:21 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]': finished 2024-04-25T00:23:22.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:21 smithi169 ceph-mon[33112]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:23:22.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:22.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/377022102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]: dispatch 2024-04-25T00:23:22.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:21 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]: dispatch 2024-04-25T00:23:22.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:21 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a"}]': finished 2024-04-25T00:23:22.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:21 smithi040 ceph-mon[32945]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:23:22.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:22 smithi031 ceph-mon[29167]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:22 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/1870290770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:23.040 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:22 smithi169 ceph-mon[33112]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:23.041 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:22 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/1870290770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:23.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:22 smithi040 ceph-mon[32945]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:23.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:22 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/1870290770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:24.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:24 smithi040 ceph-mon[32945]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:25.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:24 smithi169 ceph-mon[33112]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:25.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:24 smithi031 ceph-mon[29167]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:27.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:26 smithi031 ceph-mon[29167]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:27.071 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:26 smithi169 ceph-mon[33112]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:27.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:26 smithi040 ceph-mon[32945]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:28 smithi169 ceph-mon[33112]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:23:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:28 smithi031 ceph-mon[29167]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:23:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:29.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:28 smithi040 ceph-mon[32945]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:29.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:23:29.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:29.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:29 smithi169 ceph-mon[33112]: Deploying daemon osd.2 on smithi169 2024-04-25T00:23:30.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:29 smithi031 ceph-mon[29167]: Deploying daemon osd.2 on smithi169 2024-04-25T00:23:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:29 smithi040 ceph-mon[32945]: Deploying daemon osd.2 on smithi169 2024-04-25T00:23:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:30 smithi169 ceph-mon[33112]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:31.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:30 smithi031 ceph-mon[29167]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:31.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:30 smithi040 ceph-mon[32945]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:32.814 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:32 smithi169 ceph-mon[33112]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:33.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:32 smithi031 ceph-mon[29167]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:33.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:32 smithi040 ceph-mon[32945]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:34 smithi169 ceph-mon[33112]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:34 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:34 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:34 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:34 smithi040 ceph-mon[32945]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:34 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:34 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:34 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:34 smithi031 ceph-mon[29167]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:34 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:34 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:34 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:37.241 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:36 smithi169 ceph-mon[33112]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:37.242 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:36 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.242 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:36 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.242 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:36 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:37.242 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:36 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:37.242 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:36 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:36 smithi031 ceph-mon[29167]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:36 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.322 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.321+0000 7f12f67fc700 1 -- 172.21.15.169:0/2079355999 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1300062670 con 0x7f12ec0a2c10 2024-04-25T00:23:37.322 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 2 on host 'smithi169' 2024-04-25T00:23:37.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.323+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ec0a2c10 msgr2=0x7f12ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:37.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.323+0000 7f1307879700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ec0a2c10 0x7f12ec0a50d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005950 tx=0x7f12f000b410 comp rx=0 tx=0).stop 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 msgr2=0x7f1300198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 0x7f1300198310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00c920 tx=0x7f12fc00cc30 comp rx=0 tx=0).stop 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 shutdown_connections 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1300103ec0 0x7f130019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300100540 0x7f1300198310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ec0a2c10 0x7f12ec0a50d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 --2- 172.21.15.169:0/2079355999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300103520 0x7f1300198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 >> 172.21.15.169:0/2079355999 conn(0x7f13000fbec0 msgr2=0x7f13001068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 shutdown_connections 2024-04-25T00:23:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:23:37.324+0000 7f1307879700 1 -- 172.21.15.169:0/2079355999 wait complete. 2024-04-25T00:23:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:36 smithi040 ceph-mon[32945]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:36 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:36 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:36 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:36 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:36 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:37.954 DEBUG:teuthology.orchestra.run.smithi169:osd.2> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T00:23:37.957 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-04-25T00:23:37.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:23:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:23:38.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:38 smithi031 ceph-mon[29167]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:38.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:38.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:38.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:38.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:38 smithi031 ceph-mon[29167]: from='osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:23:38.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:38 smithi031 ceph-mon[29167]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:23:38.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:38 smithi169 ceph-mon[33112]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:38.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:38 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:38.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:38 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:38.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:38 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:38.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:38 smithi169 ceph-mon[33112]: from='osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:23:38.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:38 smithi169 ceph-mon[33112]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:23:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:38 smithi040 ceph-mon[32945]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:38 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:38 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:38 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:38 smithi040 ceph-mon[32945]: from='osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:23:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:38 smithi040 ceph-mon[32945]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:23:39.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-mon[33112]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:23:39.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-mon[33112]: from='osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:23:39.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-mon[33112]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:23:39.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:39.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-mon[33112]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:23:39.385 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2[38382]: 2024-04-25T00:23:39.287+0000 7fd7d150e700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:23:39.385 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:39 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2[38382]: 2024-04-25T00:23:39.291+0000 7fd7cd301700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:23:39.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:39 smithi031 ceph-mon[29167]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:23:39.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:39 smithi031 ceph-mon[29167]: from='osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:23:39.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:39 smithi031 ceph-mon[29167]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:23:39.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:39 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:39.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:39 smithi031 ceph-mon[29167]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:23:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:39 smithi040 ceph-mon[32945]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:23:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:39 smithi040 ceph-mon[32945]: from='osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:23:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:39 smithi040 ceph-mon[32945]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:23:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:39 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:39 smithi040 ceph-mon[32945]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:23:40.157 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:23:40.157 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:23:40.157 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T00:23:40.157 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T00:23:40.157 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133614 s, 785 MB/s 2024-04-25T00:23:40.157 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T00:23:40.541 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:40 smithi169 ceph-mon[33112]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:40.542 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:40 smithi169 ceph-mon[33112]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:23:40.542 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:40 smithi169 ceph-mon[33112]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:23:40.542 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:40 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:40.583 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:40 smithi031 ceph-mon[29167]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:40.583 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:40 smithi031 ceph-mon[29167]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:23:40.583 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:40 smithi031 ceph-mon[29167]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:23:40.583 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:40 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:40.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:40 smithi040 ceph-mon[32945]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:40.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:40 smithi040 ceph-mon[32945]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:23:40.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:40 smithi040 ceph-mon[32945]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:23:40.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:40 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:40.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-04-25T00:23:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] boot 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:41.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi169 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:41.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] boot 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi169 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T00:23:41.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.641 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:41.641 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:41.641 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: osd.2 [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] boot 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi169 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:41.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:41.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:42.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:42 smithi031 ceph-mon[29167]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:42.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:23:42.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:42 smithi031 ceph-mon[29167]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:23:42.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:42 smithi169 ceph-mon[33112]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:42.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:42 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:23:42.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:42 smithi169 ceph-mon[33112]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:23:42.644 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:42 smithi040 ceph-mon[32945]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:23:42.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:42 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:23:42.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:42 smithi040 ceph-mon[32945]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 -- 172.21.15.31:0/2205906616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 msgr2=0x7f7b6c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 --2- 172.21.15.31:0/2205906616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c10a3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b0a0 tx=0x7f7b6400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 -- 172.21.15.31:0/2205906616 shutdown_connections 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 --2- 172.21.15.31:0/2205906616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c10a3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 --2- 172.21.15.31:0/2205906616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c102d20 0x7f7b6c105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 --2- 172.21.15.31:0/2205906616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b6c105650 0x7f7b6c107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 -- 172.21.15.31:0/2205906616 >> 172.21.15.31:0/2205906616 conn(0x7f7b6c0fc730 msgr2=0x7f7b6c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 -- 172.21.15.31:0/2205906616 shutdown_connections 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.881+0000 7f7b70f27700 1 -- 172.21.15.31:0/2205906616 wait complete. 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 Processor -- start 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 -- start start 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b6c102d20 0x7f7b6c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c105650 0x7f7b6c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10dd70 con 0x7f7b6c107f80 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10dbf0 con 0x7f7b6c102d20 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b70f27700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10def0 con 0x7f7b6c105650 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c19f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48596/0 (socket says 172.21.15.31:48596) 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 -- 172.21.15.31:0/3728806700 learned_addr learned my addr 172.21.15.31:0/3728806700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:23:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6a59c700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b6c102d20 0x7f7b6c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b69d9b700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c105650 0x7f7b6c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 -- 172.21.15.31:0/3728806700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c105650 msgr2=0x7f7b6c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c105650 0x7f7b6c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 -- 172.21.15.31:0/3728806700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b6c102d20 msgr2=0x7f7b6c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b6c102d20 0x7f7b6c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.882+0000 7f7b6ad9d700 1 -- 172.21.15.31:0/3728806700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1a0070 con 0x7f7b6c107f80 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.883+0000 7f7b6ad9d700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c19f9d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006070 tx=0x7f7b64004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.883+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64021070 con 0x7f7b6c107f80 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.883+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1a0300 con 0x7f7b6c107f80 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.883+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b64007c80 con 0x7f7b6c107f80 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.883+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6401d450 con 0x7f7b6c107f80 2024-04-25T00:23:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.883+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a0880 con 0x7f7b6c107f80 2024-04-25T00:23:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.884+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c068f20 con 0x7f7b6c107f80 2024-04-25T00:23:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.885+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7b64003a00 con 0x7f7b6c107f80 2024-04-25T00:23:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.886+0000 7f7b5b7fe700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7b540a2b90 0x7f7b540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.886+0000 7f7b6a59c700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7b540a2b90 0x7f7b540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.886+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f7b6400f020 con 0x7f7b6c107f80 2024-04-25T00:23:42.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.886+0000 7f7b6a59c700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7b540a2b90 0x7f7b540a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009a10 tx=0x7f7b5c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:42.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:42.890+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7b6402a020 con 0x7f7b6c107f80 2024-04-25T00:23:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:43.109+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7b6c062670 con 0x7f7b540a2b90 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:23:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:23:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:23:43.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:44.106 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:43 smithi031 sudo[46242]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:23:44.107 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:43 smithi031 sudo[46242]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:44.107 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:43 smithi031 sudo[46242]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:44.107 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:43 smithi031 sudo[46242]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:44.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:44 smithi031 ceph-mon[29167]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:44.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:44 smithi031 ceph-mon[29167]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:23:44.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:23:44.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:44 smithi031 ceph-mon[29167]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:23:44.522 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:44 smithi031 sudo[46262]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:23:44.522 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:44 smithi031 sudo[46262]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:44.523 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:44 smithi031 sudo[46262]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:44.523 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:23:44 smithi031 sudo[46262]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:44.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:44 smithi169 ceph-mon[33112]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:44.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:44 smithi169 ceph-mon[33112]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:23:44.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:23:44.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:44 smithi169 ceph-mon[33112]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:23:44.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:44 smithi040 ceph-mon[32945]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:44.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:44 smithi040 ceph-mon[32945]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:23:44.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:23:44.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:44 smithi040 ceph-mon[32945]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:23:45.145 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:44 smithi040 sudo[41409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:23:45.145 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:44 smithi040 sudo[41409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:45.145 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:44 smithi040 sudo[41409]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:45.145 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:44 smithi040 sudo[41409]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:23:45.354+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7b64099db0 con 0x7f7b6c107f80 2024-04-25T00:23:45.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:45 smithi031 ceph-mon[29167]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T00:23:45.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:45.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:45.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:45 smithi169 ceph-mon[33112]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T00:23:45.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:45.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:45.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:23:45.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:45.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:45.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42273]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:45.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:45 smithi040 ceph-mon[32945]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T00:23:45.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:45.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:45.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:45 smithi040 sudo[41412]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:23:45.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:45 smithi040 sudo[41412]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:45.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:45 smithi040 sudo[41412]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:45.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:23:45 smithi040 sudo[41412]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:46.141 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:23:46.142 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:46.142 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42276]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:46.142 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:23:45 smithi169 sudo[42276]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:46.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46340]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:23:46.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46340]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:46.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46340]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:46.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46340]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:46.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 ceph-mon[33112]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:23:46.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 ceph-mon[33112]: Cluster is now healthy 2024-04-25T00:23:46.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 ceph-mon[33112]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:23:46.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:46.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 ceph-mon[33112]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:46.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 sudo[42279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:23:46.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 sudo[42279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:46.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 sudo[42279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:46.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:46 smithi169 sudo[42279]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46343]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46343]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 sudo[46343]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 ceph-mon[29167]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 ceph-mon[29167]: Cluster is now healthy 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 ceph-mon[29167]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:23:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:47.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:46 smithi031 ceph-mon[29167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:46 smithi040 ceph-mon[32945]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:23:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:46 smithi040 ceph-mon[32945]: Cluster is now healthy 2024-04-25T00:23:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:46 smithi040 ceph-mon[32945]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:23:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:23:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:46 smithi040 ceph-mon[32945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:47 smithi169 sudo[42282]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:23:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:47 smithi169 sudo[42282]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:47 smithi169 sudo[42282]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:47 smithi169 sudo[42282]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:48.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:47 smithi040 sudo[41415]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:23:48.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:47 smithi040 sudo[41415]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:48.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:47 smithi040 sudo[41415]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:48.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:47 smithi040 sudo[41415]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3974358295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "938ed94c-292f-4151-ad75-7a901afb1717"}]: dispatch 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3974358295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "938ed94c-292f-4151-ad75-7a901afb1717"}]': finished 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.393 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.394 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:23:48.394 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:48.394 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.394 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.394 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.394 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:48 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2803974168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 sudo[41418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 sudo[41418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 sudo[41418]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 sudo[41418]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3974358295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "938ed94c-292f-4151-ad75-7a901afb1717"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3974358295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "938ed94c-292f-4151-ad75-7a901afb1717"}]': finished 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:48 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2803974168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:48.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:48.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:48.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3974358295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "938ed94c-292f-4151-ad75-7a901afb1717"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3974358295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "938ed94c-292f-4151-ad75-7a901afb1717"}]': finished 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:23:48.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:48 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2803974168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:23:49.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:49 smithi169 ceph-mon[33112]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:49 smithi169 ceph-mon[33112]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:49 smithi169 ceph-mon[33112]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:49 smithi040 ceph-mon[32945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:49 smithi040 ceph-mon[32945]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:49 smithi040 ceph-mon[32945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:49 smithi031 ceph-mon[29167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:23:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:49 smithi031 ceph-mon[29167]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:49 smithi031 ceph-mon[29167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:23:51.367 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:51 smithi031 ceph-mon[29167]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:51.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:51 smithi169 ceph-mon[33112]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:51 smithi040 ceph-mon[32945]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:53 smithi031 ceph-mon[29167]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:53.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:53 smithi169 ceph-mon[33112]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:53 smithi040 ceph-mon[32945]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:23:55.345 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:55 smithi031 ceph-mon[29167]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-04-25T00:23:55.346 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:23:55.346 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:55.346 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:55 smithi031 ceph-mon[29167]: Deploying daemon osd.3 on smithi031 2024-04-25T00:23:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:55 smithi169 ceph-mon[33112]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-04-25T00:23:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:23:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:55 smithi169 ceph-mon[33112]: Deploying daemon osd.3 on smithi031 2024-04-25T00:23:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:55 smithi040 ceph-mon[32945]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-04-25T00:23:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:23:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:55 smithi040 ceph-mon[32945]: Deploying daemon osd.3 on smithi031 2024-04-25T00:23:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:57 smithi169 ceph-mon[33112]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-25T00:23:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:57 smithi040 ceph-mon[32945]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-25T00:23:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:57 smithi031 ceph-mon[29167]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-25T00:23:59.208 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:23:59 smithi031 ceph-mon[29167]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-25T00:23:59.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:23:59 smithi169 ceph-mon[33112]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-25T00:23:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:23:59 smithi040 ceph-mon[32945]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-25T00:24:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:00 smithi031 ceph-mon[29167]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T00:24:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:00 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:00 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:00 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:01.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:00 smithi169 ceph-mon[33112]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T00:24:01.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:00 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:01.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:00 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:01.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:00 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:00 smithi040 ceph-mon[32945]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T00:24:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:00 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:00 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:00 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:02 smithi031 ceph-mon[29167]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:24:03.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:02 smithi169 ceph-mon[33112]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:24:03.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:02 smithi040 ceph-mon[32945]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:24:03.673 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-04-25T00:24:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.672+0000 7f7b5b7fe700 1 -- 172.21.15.31:0/3728806700 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7b6c062670 con 0x7f7b540a2b90 2024-04-25T00:24:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7b540a2b90 msgr2=0x7f7b540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7b540a2b90 0x7f7b540a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009a10 tx=0x7f7b5c009190 comp rx=0 tx=0).stop 2024-04-25T00:24:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 msgr2=0x7f7b6c19f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c19f9d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006070 tx=0x7f7b64004e40 comp rx=0 tx=0).stop 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 shutdown_connections 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7b540a2b90 0x7f7b540a5050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b6c107f80 0x7f7b6c19f9d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b6c102d20 0x7f7b6c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 --2- 172.21.15.31:0/3728806700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c105650 0x7f7b6c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 >> 172.21.15.31:0/3728806700 conn(0x7f7b6c0fc730 msgr2=0x7f7b6c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 shutdown_connections 2024-04-25T00:24:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:03.675+0000 7f7b70f27700 1 -- 172.21.15.31:0/3728806700 wait complete. 2024-04-25T00:24:04.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:04.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:04.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:24:04.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:04.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:03 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:04.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:03 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:03 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:04.416 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T00:24:04.419 INFO:tasks.cephadm:Deploying osd.4 on smithi040 with /dev/vg_nvme/lv_3... 2024-04-25T00:24:04.419 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:24:04.673 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:24:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:04 smithi031 ceph-mon[29167]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:24:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:04 smithi031 ceph-mon[29167]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:24:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:04 smithi031 ceph-mon[29167]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:24:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:04 smithi031 ceph-mon[29167]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:24:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:05.029 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:04 smithi040 ceph-mon[32945]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:24:05.030 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:04 smithi040 ceph-mon[32945]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:24:05.030 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:04 smithi040 ceph-mon[32945]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:24:05.030 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:04 smithi040 ceph-mon[32945]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:24:05.030 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:05.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:04 smithi169 ceph-mon[33112]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:24:05.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:04 smithi169 ceph-mon[33112]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:24:05.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:04 smithi169 ceph-mon[33112]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:24:05.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:04 smithi169 ceph-mon[33112]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:24:05.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:05.522 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3[48377]: 2024-04-25T00:24:05.221+0000 7f7ba1125700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:24:05.522 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3[48377]: 2024-04-25T00:24:05.229+0000 7f7b98f10700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:24:05.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-mon[29167]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:24:05.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-mon[29167]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T00:24:05.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:05.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:05.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:05 smithi031 ceph-mon[29167]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' 2024-04-25T00:24:06.057 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:05 smithi040 ceph-mon[32945]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:24:06.057 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:05 smithi040 ceph-mon[32945]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T00:24:06.057 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:06.058 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:06.058 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:05 smithi040 ceph-mon[32945]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' 2024-04-25T00:24:06.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:05 smithi169 ceph-mon[33112]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:24:06.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:05 smithi169 ceph-mon[33112]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T00:24:06.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:06.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:06.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:05 smithi169 ceph-mon[33112]: from='osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832]' entity='osd.3' 2024-04-25T00:24:06.654 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:24:06.655 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:24:06.655 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T00:24:06.655 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T00:24:06.655 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109236 s, 960 MB/s 2024-04-25T00:24:06.655 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T00:24:07.031 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:24:07.031 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:24:07.031 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:24:07.031 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:07.031 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] boot 2024-04-25T00:24:07.032 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:24:07.032 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:06 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] boot 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:24:07.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:06 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:07.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:24:07.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:24:07.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:24:07.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:07.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: osd.3 [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] boot 2024-04-25T00:24:07.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:24:07.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:06 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:24:07.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_3 2024-04-25T00:24:07.607 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:24:08.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:08.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:24:08.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:08.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:08.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:08.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:08.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:08.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:09.058 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:08 smithi040 ceph-mon[32945]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:09.058 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:08 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi031 2024-04-25T00:24:09.058 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:08 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi031 to 8546M 2024-04-25T00:24:09.059 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:08 smithi040 ceph-mon[32945]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:24:09.059 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:08 smithi040 ceph-mon[32945]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:24:09.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 -- 172.21.15.40:0/211698242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 msgr2=0x7f67a41051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 --2- 172.21.15.40:0/211698242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a41051f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6794009990 tx=0x7f6794009ca0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 -- 172.21.15.40:0/211698242 shutdown_connections 2024-04-25T00:24:09.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 --2- 172.21.15.40:0/211698242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a4108060 0x7f67a410a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 --2- 172.21.15.40:0/211698242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a41051f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 --2- 172.21.15.40:0/211698242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a4105730 0x7f67a4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.069+0000 7f67aa188700 1 -- 172.21.15.40:0/211698242 >> 172.21.15.40:0/211698242 conn(0x7f67a40fc750 msgr2=0x7f67a40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 -- 172.21.15.40:0/211698242 shutdown_connections 2024-04-25T00:24:09.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 -- 172.21.15.40:0/211698242 wait complete. 2024-04-25T00:24:09.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 Processor -- start 2024-04-25T00:24:09.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 -- start start 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a4105730 0x7f67a419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a4108060 0x7f67a41a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67a410dd80 con 0x7f67a4108060 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67a410dc00 con 0x7f67a4102e00 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.070+0000 7f67aa188700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f67a410df00 con 0x7f67a4105730 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34724/0 (socket says 172.21.15.40:34724) 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 -- 172.21.15.40:0/3358330056 learned_addr learned my addr 172.21.15.40:0/3358330056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a3fff700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a4108060 0x7f67a41a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a2ffd700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a4105730 0x7f67a419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 -- 172.21.15.40:0/3358330056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a4105730 msgr2=0x7f67a419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a4105730 0x7f67a419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 -- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a4108060 msgr2=0x7f67a41a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a4108060 0x7f67a41a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 -- 172.21.15.40:0/3358330056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a41a21a0 con 0x7f67a4102e00 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a37fe700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a419a540 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6794000c00 tx=0x7f67940047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794027070 con 0x7f67a4102e00 2024-04-25T00:24:09.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41a2430 con 0x7f67a4102e00 2024-04-25T00:24:09.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6794022470 con 0x7f67a4102e00 2024-04-25T00:24:09.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679400f6c0 con 0x7f67a4102e00 2024-04-25T00:24:09.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.071+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41a28e0 con 0x7f67a4102e00 2024-04-25T00:24:09.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.072+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a4068f20 con 0x7f67a4102e00 2024-04-25T00:24:09.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.073+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f679402a070 con 0x7f67a4102e00 2024-04-25T00:24:09.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.074+0000 7f67a0ff9700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f678c0a2c20 0x7f678c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.074+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f67940cf4c0 con 0x7f67a4102e00 2024-04-25T00:24:09.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.074+0000 7f67a2ffd700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f678c0a2c20 0x7f678c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.075+0000 7f67a2ffd700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f678c0a2c20 0x7f678c0a50e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f679800ba60 tx=0x7f6798005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.078+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6794031070 con 0x7f67a4102e00 2024-04-25T00:24:09.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:08 smithi169 ceph-mon[33112]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:09.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:08 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi031 2024-04-25T00:24:09.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:08 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi031 to 8546M 2024-04-25T00:24:09.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:08 smithi169 ceph-mon[33112]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:24:09.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:08 smithi169 ceph-mon[33112]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:24:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:08 smithi031 ceph-mon[29167]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:08 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi031 2024-04-25T00:24:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:08 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi031 to 8546M 2024-04-25T00:24:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:08 smithi031 ceph-mon[29167]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:24:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:08 smithi031 ceph-mon[29167]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:24:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:09.295+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f67a4062670 con 0x7f678c0a2c20 2024-04-25T00:24:10.245 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:10 smithi040 ceph-mon[32945]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T00:24:10.245 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:24:10.245 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:24:10.245 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:10.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:10 smithi031 ceph-mon[29167]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T00:24:10.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:24:10.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:24:10.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:10.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:10 smithi169 ceph-mon[33112]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T00:24:10.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:24:10.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:24:10.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:11.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:11 smithi040 ceph-mon[32945]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:11.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:11 smithi040 ceph-mon[32945]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:24:11.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:11 smithi169 ceph-mon[33112]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:11.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:11 smithi169 ceph-mon[33112]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:24:11.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:11 smithi031 ceph-mon[29167]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:11.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:11 smithi031 ceph-mon[29167]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:24:12.381 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:12 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/2527660250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]: dispatch 2024-04-25T00:24:12.381 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:12 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]: dispatch 2024-04-25T00:24:12.381 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:12 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]': finished 2024-04-25T00:24:12.381 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:12 smithi040 ceph-mon[32945]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:24:12.382 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:12.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:12 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/2527660250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]: dispatch 2024-04-25T00:24:12.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:12 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]: dispatch 2024-04-25T00:24:12.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:12 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]': finished 2024-04-25T00:24:12.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:12 smithi169 ceph-mon[33112]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:24:12.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:12 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/2527660250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]: dispatch 2024-04-25T00:24:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:12 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]: dispatch 2024-04-25T00:24:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:12 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d02417-522a-472c-bc86-eb4a65662c6c"}]': finished 2024-04-25T00:24:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:12 smithi031 ceph-mon[29167]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:24:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:13 smithi169 ceph-mon[33112]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:13 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/4179595197' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:24:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:13 smithi040 ceph-mon[32945]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:13 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/4179595197' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:24:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:13 smithi031 ceph-mon[29167]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:13 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/4179595197' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:24:15.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:15 smithi169 ceph-mon[33112]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:24:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:15 smithi040 ceph-mon[32945]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:24:15.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:15 smithi031 ceph-mon[29167]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:24:17.382 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:17 smithi040 ceph-mon[32945]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:24:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:17 smithi169 ceph-mon[33112]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:24:17.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:17 smithi031 ceph-mon[29167]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:24:19.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:19 smithi169 ceph-mon[33112]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:24:19.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:19 smithi040 ceph-mon[32945]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:24:19.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:19 smithi031 ceph-mon[29167]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:24:20.355 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:24:20.355 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:20.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:24:20.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:20.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:24:20.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:21.294 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:21 smithi040 ceph-mon[32945]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:24:21.294 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:21 smithi040 ceph-mon[32945]: Deploying daemon osd.4 on smithi040 2024-04-25T00:24:21.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:21 smithi169 ceph-mon[33112]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:24:21.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:21 smithi169 ceph-mon[33112]: Deploying daemon osd.4 on smithi040 2024-04-25T00:24:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:21 smithi031 ceph-mon[29167]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:24:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:21 smithi031 ceph-mon[29167]: Deploying daemon osd.4 on smithi040 2024-04-25T00:24:23.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:23 smithi040 ceph-mon[32945]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:24:23.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:23 smithi169 ceph-mon[33112]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:24:23.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:23 smithi031 ceph-mon[29167]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:24:25.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:25 smithi169 ceph-mon[33112]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T00:24:25.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:25 smithi040 ceph-mon[32945]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T00:24:25.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:25 smithi031 ceph-mon[29167]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T00:24:26.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:26 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:26.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:26 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:26.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:26 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:26.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:26 smithi040 ceph-mon[32945]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:26.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:26.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:26.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:26.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:26 smithi031 ceph-mon[29167]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:26.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:26 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:26.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:26 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:26.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:26 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:26.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:26 smithi169 ceph-mon[33112]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:29.016 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 4 on host 'smithi040' 2024-04-25T00:24:29.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.015+0000 7f67a0ff9700 1 -- 172.21.15.40:0/3358330056 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67a4062670 con 0x7f678c0a2c20 2024-04-25T00:24:29.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f678c0a2c20 msgr2=0x7f678c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:29.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f678c0a2c20 0x7f678c0a50e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f679800ba60 tx=0x7f6798005d50 comp rx=0 tx=0).stop 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 msgr2=0x7f67a419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a419a540 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6794000c00 tx=0x7f67940047b0 comp rx=0 tx=0).stop 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 shutdown_connections 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f678c0a2c20 0x7f678c0a50e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a4108060 0x7f67a41a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67a4102e00 0x7f67a419a540 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 --2- 172.21.15.40:0/3358330056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a4105730 0x7f67a419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 >> 172.21.15.40:0/3358330056 conn(0x7f67a40fc750 msgr2=0x7f67a4102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 shutdown_connections 2024-04-25T00:24:29.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:24:29.017+0000 7f67aa188700 1 -- 172.21.15.40:0/3358330056 wait complete. 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:29.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:29.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:29.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:29.619 DEBUG:teuthology.orchestra.run.smithi040:osd.4> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T00:24:29.621 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_3... 2024-04-25T00:24:29.622 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:24:29.868 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:24:30.308 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:30 smithi169 ceph-mon[33112]: from='osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:24:30.309 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:30 smithi169 ceph-mon[33112]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:24:30.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:30 smithi040 ceph-mon[32945]: from='osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:24:30.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:30 smithi040 ceph-mon[32945]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:24:30.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:30 smithi031 ceph-mon[29167]: from='osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:24:30.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:30 smithi031 ceph-mon[29167]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:24:31.137 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-mon[32945]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:31.137 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-mon[32945]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:24:31.137 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-mon[32945]: from='osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:24:31.137 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-mon[32945]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:24:31.137 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:31.137 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-mon[32945]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:24:31.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:31 smithi169 ceph-mon[33112]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:31.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:31 smithi169 ceph-mon[33112]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:24:31.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:31 smithi169 ceph-mon[33112]: from='osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:24:31.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:31 smithi169 ceph-mon[33112]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:24:31.267 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:31.268 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:31 smithi169 ceph-mon[33112]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:24:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:31 smithi031 ceph-mon[29167]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:31 smithi031 ceph-mon[29167]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:24:31.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:31 smithi031 ceph-mon[29167]: from='osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:24:31.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:31 smithi031 ceph-mon[29167]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:24:31.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:31.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:31 smithi031 ceph-mon[29167]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:24:31.814 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4[44210]: 2024-04-25T00:24:31.526+0000 7fb3be4c2700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:24:31.815 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:24:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4[44210]: 2024-04-25T00:24:31.532+0000 7fb3b5aac700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:24:31.960 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:24:31.960 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:24:31.960 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T00:24:31.960 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T00:24:31.960 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011514 s, 911 MB/s 2024-04-25T00:24:31.960 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T00:24:32.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:32 smithi169 ceph-mon[33112]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:24:32.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:32 smithi169 ceph-mon[33112]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T00:24:32.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:32 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:32.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:32 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:32.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:32 smithi169 ceph-mon[33112]: from='osd.4 ' entity='osd.4' 2024-04-25T00:24:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:32 smithi040 ceph-mon[32945]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:24:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:32 smithi040 ceph-mon[32945]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T00:24:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:32 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:32 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:32.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:32 smithi040 ceph-mon[32945]: from='osd.4 ' entity='osd.4' 2024-04-25T00:24:32.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:32 smithi031 ceph-mon[29167]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:24:32.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:32 smithi031 ceph-mon[29167]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T00:24:32.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:32 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:32.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:32 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:32.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:32 smithi031 ceph-mon[29167]: from='osd.4 ' entity='osd.4' 2024-04-25T00:24:32.673 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-04-25T00:24:32.918 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] boot 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.290 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.291 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:33.291 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:33.291 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.291 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:33.291 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:33.291 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:24:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:24:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] boot 2024-04-25T00:24:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:33.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:24:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:24:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: osd.4 [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] boot 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:33.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:34.346 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:34 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi040 2024-04-25T00:24:34.346 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:34 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi040 to 8546M 2024-04-25T00:24:34.346 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:34 smithi169 ceph-mon[33112]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T00:24:34.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:34 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi040 2024-04-25T00:24:34.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:34 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi040 to 8546M 2024-04-25T00:24:34.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:34 smithi040 ceph-mon[32945]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T00:24:34.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:34 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi040 2024-04-25T00:24:34.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:34 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi040 to 8546M 2024-04-25T00:24:34.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:34 smithi031 ceph-mon[29167]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 -- 172.21.15.169:0/2526573963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100d20 msgr2=0x7efd4c1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 --2- 172.21.15.169:0/2526573963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100d20 0x7efd4c1011d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd4400b0a0 tx=0x7efd4400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 -- 172.21.15.169:0/2526573963 shutdown_connections 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 --2- 172.21.15.169:0/2526573963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1027f0 0x7efd4c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 --2- 172.21.15.169:0/2526573963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100d20 0x7efd4c1011d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 --2- 172.21.15.169:0/2526573963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c100380 0x7efd4c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 -- 172.21.15.169:0/2526573963 >> 172.21.15.169:0/2526573963 conn(0x7efd4c0fbf00 msgr2=0x7efd4c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 -- 172.21.15.169:0/2526573963 shutdown_connections 2024-04-25T00:24:34.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.560+0000 7efd51919700 1 -- 172.21.15.169:0/2526573963 wait complete. 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 Processor -- start 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 -- start start 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 0x7efd4c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c100d20 0x7efd4c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c1027f0 0x7efd4c1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10ff80 con 0x7efd4c100d20 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10fe00 con 0x7efd4c1027f0 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd51919700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c110100 con 0x7efd4c100380 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd4affd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 0x7efd4c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd4affd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 0x7efd4c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46762/0 (socket says 172.21.15.169:46762) 2024-04-25T00:24:34.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd4affd700 1 -- 172.21.15.169:0/2271549726 learned_addr learned my addr 172.21.15.169:0/2271549726 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.561+0000 7efd427fc700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c100d20 0x7efd4c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4b7fe700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c1027f0 0x7efd4c1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4affd700 1 -- 172.21.15.169:0/2271549726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c1027f0 msgr2=0x7efd4c1a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4affd700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c1027f0 0x7efd4c1a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4affd700 1 -- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c100d20 msgr2=0x7efd4c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4affd700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c100d20 0x7efd4c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4affd700 1 -- 172.21.15.169:0/2271549726 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c1a2220 con 0x7efd4c100380 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4b7fe700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c1027f0 0x7efd4c1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd4affd700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 0x7efd4c19a5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00d9b0 tx=0x7efd3c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c016070 con 0x7efd4c100380 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd3c00be00 con 0x7efd4c100380 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c1a2510 con 0x7efd4c100380 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c00f3c0 con 0x7efd4c100380 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.562+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c1a2aa0 con 0x7efd4c100380 2024-04-25T00:24:34.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.563+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd4c068f20 con 0x7efd4c100380 2024-04-25T00:24:34.566 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.564+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efd3c019070 con 0x7efd4c100380 2024-04-25T00:24:34.566 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.565+0000 7efd50917700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd380a2ca0 0x7efd380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:34.566 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.565+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7efd3c0cdd50 con 0x7efd4c100380 2024-04-25T00:24:34.566 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.565+0000 7efd427fc700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd380a2ca0 0x7efd380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:34.567 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.565+0000 7efd427fc700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd380a2ca0 0x7efd380a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd34004630 tx=0x7efd3400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:34.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.569+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd3c0d1030 con 0x7efd4c100380 2024-04-25T00:24:34.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:34.777+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efd4c062670 con 0x7efd380a2ca0 2024-04-25T00:24:35.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:35 smithi169 ceph-mon[33112]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:35.385 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:24:35.386 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:24:35.386 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:35 smithi040 ceph-mon[32945]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:24:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:24:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:35 smithi031 ceph-mon[29167]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:24:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:24:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:36.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:36 smithi169 ceph-mon[33112]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:24:36.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:36 smithi040 ceph-mon[32945]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:24:36.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:36 smithi031 ceph-mon[29167]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:24:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:37 smithi169 ceph-mon[33112]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:37 smithi040 ceph-mon[32945]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:37 smithi031 ceph-mon[29167]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:38.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:38 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/3384665778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]: dispatch 2024-04-25T00:24:38.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:38 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]: dispatch 2024-04-25T00:24:38.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:38 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]': finished 2024-04-25T00:24:38.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:38 smithi169 ceph-mon[33112]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:24:38.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:38 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:38.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:38 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/3384665778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]: dispatch 2024-04-25T00:24:38.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:38 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]: dispatch 2024-04-25T00:24:38.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:38 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]': finished 2024-04-25T00:24:38.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:38 smithi040 ceph-mon[32945]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:24:38.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:38 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:38 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/3384665778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]: dispatch 2024-04-25T00:24:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:38 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]: dispatch 2024-04-25T00:24:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:38 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c51c2807-c67a-46ef-8add-0ee8009cdd3e"}]': finished 2024-04-25T00:24:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:38 smithi031 ceph-mon[29167]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:24:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:39 smithi169 ceph-mon[33112]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:39.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:39 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/2224307390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:24:39.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:39 smithi040 ceph-mon[32945]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:39 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/2224307390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:24:39.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:39 smithi031 ceph-mon[29167]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:39.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:39 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/2224307390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:24:41.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:41 smithi169 ceph-mon[33112]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:41.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:41 smithi040 ceph-mon[32945]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:41 smithi031 ceph-mon[29167]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:43.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:43 smithi169 ceph-mon[33112]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:43.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:43 smithi040 ceph-mon[32945]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:43 smithi031 ceph-mon[29167]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:45.368 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:45 smithi169 ceph-mon[33112]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:45.368 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:24:45.368 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:45 smithi040 ceph-mon[32945]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:24:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:45 smithi031 ceph-mon[29167]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:24:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:46.186 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:46 smithi169 ceph-mon[33112]: Deploying daemon osd.5 on smithi169 2024-04-25T00:24:46.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:46 smithi040 ceph-mon[32945]: Deploying daemon osd.5 on smithi169 2024-04-25T00:24:46.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:46 smithi031 ceph-mon[29167]: Deploying daemon osd.5 on smithi169 2024-04-25T00:24:47.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:47 smithi169 ceph-mon[33112]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:47 smithi040 ceph-mon[32945]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:47 smithi031 ceph-mon[29167]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:49 smithi169 ceph-mon[33112]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:49 smithi040 ceph-mon[32945]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:49 smithi031 ceph-mon[29167]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:51.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:51 smithi169 ceph-mon[33112]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:51.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:51.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:51.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:51 smithi040 ceph-mon[32945]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:51 smithi031 ceph-mon[29167]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:53.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:53 smithi169 ceph-mon[33112]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:53.339 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.339 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.339 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:53.339 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:53.339 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:53 smithi040 ceph-mon[32945]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:53 smithi031 ceph-mon[29167]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:53.798 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-04-25T00:24:53.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.797+0000 7efd50917700 1 -- 172.21.15.169:0/2271549726 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd4c062670 con 0x7efd380a2ca0 2024-04-25T00:24:53.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd380a2ca0 msgr2=0x7efd380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:53.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd380a2ca0 0x7efd380a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd34004630 tx=0x7efd3400b3c0 comp rx=0 tx=0).stop 2024-04-25T00:24:53.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 msgr2=0x7efd4c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:53.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 0x7efd4c19a5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00d9b0 tx=0x7efd3c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:24:53.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 shutdown_connections 2024-04-25T00:24:53.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4c1027f0 0x7efd4c1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:53.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c100380 0x7efd4c19a5c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:53.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd380a2ca0 0x7efd380a5160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:53.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 --2- 172.21.15.169:0/2271549726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c100d20 0x7efd4c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:53.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 >> 172.21.15.169:0/2271549726 conn(0x7efd4c0fbf00 msgr2=0x7efd4c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:53.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 shutdown_connections 2024-04-25T00:24:53.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:24:53.799+0000 7efd51919700 1 -- 172.21.15.169:0/2271549726 wait complete. 2024-04-25T00:24:54.505 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T00:24:54.507 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-04-25T00:24:54.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:24:54.763 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:24:54.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:54 smithi031 ceph-mon[29167]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:54.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:54.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:54.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:54.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:54 smithi031 ceph-mon[29167]: from='osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:24:54.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:54 smithi031 ceph-mon[29167]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:24:55.073 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:54 smithi169 ceph-mon[33112]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:55.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:55.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:55.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:55.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:54 smithi169 ceph-mon[33112]: from='osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:24:55.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:54 smithi169 ceph-mon[33112]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:24:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:54 smithi040 ceph-mon[32945]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:54 smithi040 ceph-mon[32945]: from='osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:24:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:54 smithi040 ceph-mon[32945]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:24:56.069 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:55 smithi169 ceph-mon[33112]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:24:56.069 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:55 smithi169 ceph-mon[33112]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:24:56.069 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:56.069 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:55 smithi169 ceph-mon[33112]: from='osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:24:56.069 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:55 smithi169 ceph-mon[33112]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:24:56.145 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:55 smithi031 ceph-mon[29167]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:55 smithi031 ceph-mon[29167]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:55 smithi031 ceph-mon[29167]: from='osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:55 smithi031 ceph-mon[29167]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:55 smithi040 ceph-mon[32945]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:55 smithi040 ceph-mon[32945]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:24:56.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:56.147 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:55 smithi040 ceph-mon[32945]: from='osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:24:56.147 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:55 smithi040 ceph-mon[32945]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:24:56.389 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:24:56.268+0000 7fc621c9e700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:24:56.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:24:56.275+0000 7fc619a89700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:24:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:24:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:24:56.856 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T00:24:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T00:24:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112263 s, 934 MB/s 2024-04-25T00:24:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T00:24:57.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: from='osd.5 ' entity='osd.5' 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] boot 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:24:57.162 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: from='osd.5 ' entity='osd.5' 2024-04-25T00:24:57.163 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] boot 2024-04-25T00:24:57.164 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:24:57.164 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:24:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: from='osd.5 ' entity='osd.5' 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: osd.5 [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] boot 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:24:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:24:57.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-04-25T00:24:57.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:24:57.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:57.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:57 smithi031 ceph-mon[29167]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:24:57.940 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:24:58.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:57 smithi169 ceph-mon[33112]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:24:58.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:57 smithi040 ceph-mon[32945]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:24:58.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:58 smithi031 ceph-mon[29167]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:24:58.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:58 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi169 2024-04-25T00:24:58.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:58 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T00:24:59.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:58 smithi169 ceph-mon[33112]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:24:59.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:58 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi169 2024-04-25T00:24:59.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:58 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T00:24:59.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:58 smithi040 ceph-mon[32945]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:24:59.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:58 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi169 2024-04-25T00:24:59.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:58 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T00:24:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 -- 172.21.15.31:0/2465001042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 msgr2=0x7ff0001038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 --2- 172.21.15.31:0/2465001042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001038f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feff00099c0 tx=0x7feff0009cd0 comp rx=0 tx=0).stop 2024-04-25T00:24:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 -- 172.21.15.31:0/2465001042 shutdown_connections 2024-04-25T00:24:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 --2- 172.21.15.31:0/2465001042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff000103e30 0x7ff00010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 --2- 172.21.15.31:0/2465001042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff000102290 0x7ff0001026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 --2- 172.21.15.31:0/2465001042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001038f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 -- 172.21.15.31:0/2465001042 >> 172.21.15.31:0/2465001042 conn(0x7ff0000fd830 msgr2=0x7ff0000ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 -- 172.21.15.31:0/2465001042 shutdown_connections 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 -- 172.21.15.31:0/2465001042 wait complete. 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.448+0000 7ff0075fd700 1 Processor -- start 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 -- start start 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff000102290 0x7ff0001983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff000103e30 0x7ff00019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010dd20 con 0x7ff000103490 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010dba0 con 0x7ff000103e30 2024-04-25T00:24:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff0075fd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010dea0 con 0x7ff000102290 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60284/0 (socket says 172.21.15.31:60284) 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff005399700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff000102290 0x7ff0001983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 -- 172.21.15.31:0/3564578221 learned_addr learned my addr 172.21.15.31:0/3564578221 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff005b9a700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff000103e30 0x7ff00019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 -- 172.21.15.31:0/3564578221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff000102290 msgr2=0x7ff0001983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff000102290 0x7ff0001983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 -- 172.21.15.31:0/3564578221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff000103e30 msgr2=0x7ff00019f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff000103e30 0x7ff00019f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 -- 172.21.15.31:0/3564578221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0001a0010 con 0x7ff000103490 2024-04-25T00:24:59.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.449+0000 7ff004b98700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001988f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feff0000c00 tx=0x7feff00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.450+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0027070 con 0x7ff000103490 2024-04-25T00:24:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.450+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feff0004050 con 0x7ff000103490 2024-04-25T00:24:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.450+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0001a02a0 con 0x7ff000103490 2024-04-25T00:24:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.450+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0022cb0 con 0x7ff000103490 2024-04-25T00:24:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.450+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a0750 con 0x7ff000103490 2024-04-25T00:24:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.451+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff000068f20 con 0x7ff000103490 2024-04-25T00:24:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.452+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feff002a070 con 0x7ff000103490 2024-04-25T00:24:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.452+0000 7feff67fc700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fefec0a2ca0 0x7fefec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.453+0000 7ff005399700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fefec0a2ca0 0x7fefec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.453+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7feff00cf4f0 con 0x7ff000103490 2024-04-25T00:24:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.453+0000 7ff005399700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fefec0a2ca0 0x7fefec0a5160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0001032f0 tx=0x7feffc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.456+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feff0031070 con 0x7ff000103490 2024-04-25T00:24:59.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:24:59.672+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff000062670 con 0x7fefec0a2ca0 2024-04-25T00:25:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:59 smithi031 ceph-mon[29167]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:25:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:59 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:59 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:00.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:24:59 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:00.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:59 smithi169 ceph-mon[33112]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:25:00.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:59 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:00.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:59 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:00.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:24:59 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:00.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:59 smithi040 ceph-mon[32945]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:25:00.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:59 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:00.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:59 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:00.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:24:59 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:00 smithi031 ceph-mon[29167]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:00 smithi031 ceph-mon[29167]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:00 smithi031 ceph-mon[29167]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:25:01.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:00 smithi169 ceph-mon[33112]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:01.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:00 smithi169 ceph-mon[33112]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:01.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:00 smithi169 ceph-mon[33112]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:25:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:00 smithi040 ceph-mon[32945]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:00 smithi040 ceph-mon[32945]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:00 smithi040 ceph-mon[32945]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:25:02.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:01 smithi169 ceph-mon[33112]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T00:25:02.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:01 smithi040 ceph-mon[32945]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T00:25:02.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:01 smithi031 ceph-mon[29167]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T00:25:03.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:02 smithi169 ceph-mon[33112]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:03.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:02 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/170694306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4034099-a0bd-4ef2-95a1-3b00255b587e"}]: dispatch 2024-04-25T00:25:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:02 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/170694306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4034099-a0bd-4ef2-95a1-3b00255b587e"}]': finished 2024-04-25T00:25:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:02 smithi169 ceph-mon[33112]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:25:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:02 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:02 smithi040 ceph-mon[32945]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:02 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/170694306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4034099-a0bd-4ef2-95a1-3b00255b587e"}]: dispatch 2024-04-25T00:25:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:02 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/170694306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4034099-a0bd-4ef2-95a1-3b00255b587e"}]': finished 2024-04-25T00:25:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:02 smithi040 ceph-mon[32945]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:25:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:02 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:02 smithi031 ceph-mon[29167]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:02 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/170694306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4034099-a0bd-4ef2-95a1-3b00255b587e"}]: dispatch 2024-04-25T00:25:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:02 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/170694306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4034099-a0bd-4ef2-95a1-3b00255b587e"}]': finished 2024-04-25T00:25:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:02 smithi031 ceph-mon[29167]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:25:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:02 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:04.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:03 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/604174662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:04.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:03 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/604174662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:04.173 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:03 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/604174662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:05.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:04 smithi169 ceph-mon[33112]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T00:25:05.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:04 smithi040 ceph-mon[32945]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T00:25:05.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:04 smithi031 ceph-mon[29167]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T00:25:07.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:06 smithi169 ceph-mon[33112]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:25:07.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:06 smithi040 ceph-mon[32945]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:25:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:06 smithi031 ceph-mon[29167]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:25:08.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:08 smithi031 ceph-mon[29167]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T00:25:09.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:08 smithi169 ceph-mon[33112]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T00:25:09.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:08 smithi040 ceph-mon[32945]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T00:25:11.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:10 smithi169 ceph-mon[33112]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T00:25:11.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:25:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:11.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:10 smithi040 ceph-mon[32945]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T00:25:11.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:25:11.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:11.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:10 smithi031 ceph-mon[29167]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T00:25:11.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:25:11.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:12.112 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:11 smithi031 ceph-mon[29167]: Deploying daemon osd.6 on smithi031 2024-04-25T00:25:12.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:11 smithi169 ceph-mon[33112]: Deploying daemon osd.6 on smithi031 2024-04-25T00:25:12.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:11 smithi040 ceph-mon[32945]: Deploying daemon osd.6 on smithi031 2024-04-25T00:25:13.138 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:12 smithi031 ceph-mon[29167]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:25:13.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:12 smithi169 ceph-mon[33112]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:25:13.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:12 smithi040 ceph-mon[32945]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:25:15.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:14 smithi169 ceph-mon[33112]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T00:25:15.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:14 smithi040 ceph-mon[32945]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T00:25:15.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:14 smithi031 ceph-mon[29167]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T00:25:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:16 smithi031 ceph-mon[29167]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:17.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:16 smithi169 ceph-mon[33112]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:17.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:17.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:17.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:17.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:16 smithi040 ceph-mon[32945]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:17.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:17.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:17.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:18 smithi031 ceph-mon[29167]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:19.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:18 smithi169 ceph-mon[33112]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:19.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:18 smithi040 ceph-mon[32945]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:20.589 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-04-25T00:25:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.588+0000 7feff67fc700 1 -- 172.21.15.31:0/3564578221 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff000062670 con 0x7fefec0a2ca0 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fefec0a2ca0 msgr2=0x7fefec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fefec0a2ca0 0x7fefec0a5160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0001032f0 tx=0x7feffc00b410 comp rx=0 tx=0).stop 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 msgr2=0x7ff0001988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001988f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feff0000c00 tx=0x7feff00047b0 comp rx=0 tx=0).stop 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 shutdown_connections 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff000103e30 0x7ff00019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff000102290 0x7ff0001983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fefec0a2ca0 0x7fefec0a5160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 --2- 172.21.15.31:0/3564578221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff000103490 0x7ff0001988f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 >> 172.21.15.31:0/3564578221 conn(0x7ff0000fd830 msgr2=0x7ff000106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 shutdown_connections 2024-04-25T00:25:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:25:20.591+0000 7ff0075fd700 1 -- 172.21.15.31:0/3564578221 wait complete. 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:21.292 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T00:25:21.294 INFO:tasks.cephadm:Deploying osd.7 on smithi040 with /dev/vg_nvme/lv_2... 2024-04-25T00:25:21.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:25:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:25:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:21 smithi040 ceph-mon[32945]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:25:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:21 smithi040 ceph-mon[32945]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:25:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:21 smithi040 ceph-mon[32945]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:25:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:21.827 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:21.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-mon[29167]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:25:21.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-mon[29167]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:25:21.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-mon[29167]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:25:21.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:21.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:21.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:21 smithi169 ceph-mon[33112]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:25:21.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:21 smithi169 ceph-mon[33112]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:25:21.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:21 smithi169 ceph-mon[33112]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:25:21.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:22.214 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:25:21.873+0000 7f0a33248700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:25:22.215 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:25:21 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:25:21.882+0000 7f0a2f03b700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:25:23.134 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:22 smithi040 ceph-mon[32945]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:23.134 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:22 smithi040 ceph-mon[32945]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:25:23.134 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:22 smithi040 ceph-mon[32945]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T00:25:23.134 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:23.135 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:23.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:22 smithi169 ceph-mon[33112]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:23.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:22 smithi169 ceph-mon[33112]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:25:23.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:22 smithi169 ceph-mon[33112]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T00:25:23.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:23.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:22 smithi031 ceph-mon[29167]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:22 smithi031 ceph-mon[29167]: from='osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:25:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:22 smithi031 ceph-mon[29167]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T00:25:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:25:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:25:23.540 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T00:25:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T00:25:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.010991 s, 954 MB/s 2024-04-25T00:25:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T00:25:24.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:23 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:25:24.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:23 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:25:24.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:23 smithi031 ceph-mon[29167]: osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] boot 2024-04-25T00:25:24.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:23 smithi031 ceph-mon[29167]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:25:24.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:24.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:23 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:25:24.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:23 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:23 smithi040 ceph-mon[32945]: osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] boot 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:23 smithi040 ceph-mon[32945]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:23 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:23 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:23 smithi169 ceph-mon[33112]: osd.6 [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] boot 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:23 smithi169 ceph-mon[33112]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:25:24.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:25:24.167 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_2 2024-04-25T00:25:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:25:25.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:25.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:25.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:25.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:26.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 -- 172.21.15.40:0/3302222438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28102d40 msgr2=0x7fca28105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 --2- 172.21.15.40:0/3302222438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28102d40 0x7fca28105130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca18009990 tx=0x7fca18009ca0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 -- 172.21.15.40:0/3302222438 shutdown_connections 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 --2- 172.21.15.40:0/3302222438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28107fa0 0x7fca2810a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 --2- 172.21.15.40:0/3302222438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28102d40 0x7fca28105130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 --2- 172.21.15.40:0/3302222438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28105670 0x7fca28107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 -- 172.21.15.40:0/3302222438 >> 172.21.15.40:0/3302222438 conn(0x7fca280fc750 msgr2=0x7fca280feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 -- 172.21.15.40:0/3302222438 shutdown_connections 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 -- 172.21.15.40:0/3302222438 wait complete. 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.069+0000 7fca2f343700 1 Processor -- start 2024-04-25T00:25:26.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 -- start start 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28102d40 0x7fca281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28105670 0x7fca281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 0x7fca2819f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca2810dd40 con 0x7fca28102d40 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca2810dbc0 con 0x7fca28107fa0 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2f343700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca2810dec0 con 0x7fca28105670 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 0x7fca2819f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 0x7fca2819f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46734/0 (socket says 172.21.15.40:46734) 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 -- 172.21.15.40:0/1435818119 learned_addr learned my addr 172.21.15.40:0/1435818119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2c8de700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28105670 0x7fca281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d0df700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28102d40 0x7fca281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 -- 172.21.15.40:0/1435818119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28105670 msgr2=0x7fca281988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28105670 0x7fca281988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 -- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28102d40 msgr2=0x7fca281983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28102d40 0x7fca281983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2d8e0700 1 -- 172.21.15.40:0/1435818119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca281a0010 con 0x7fca28107fa0 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.070+0000 7fca2c8de700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28105670 0x7fca281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.071+0000 7fca2d8e0700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 0x7fca2819f970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca20007ec0 tx=0x7fca2000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.071+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004bb0 con 0x7fca28107fa0 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.071+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca281a0300 con 0x7fca28107fa0 2024-04-25T00:25:26.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.071+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca20004d50 con 0x7fca28107fa0 2024-04-25T00:25:26.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.071+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004540 con 0x7fca28107fa0 2024-04-25T00:25:26.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.071+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281a0890 con 0x7fca28107fa0 2024-04-25T00:25:26.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.072+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca28068f20 con 0x7fca28107fa0 2024-04-25T00:25:26.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.073+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca2001a030 con 0x7fca28107fa0 2024-04-25T00:25:26.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.073+0000 7fca1e7fc700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fca140a2c20 0x7fca140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.074+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fca200cd9d0 con 0x7fca28107fa0 2024-04-25T00:25:26.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.074+0000 7fca2d0df700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fca140a2c20 0x7fca140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.074+0000 7fca2d0df700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fca140a2c20 0x7fca140a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b5c0 tx=0x7fca18003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.078+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fca20021020 con 0x7fca28107fa0 2024-04-25T00:25:26.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:25 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi031 2024-04-25T00:25:26.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:25 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi031 to 5697M 2024-04-25T00:25:26.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:25 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi031 2024-04-25T00:25:26.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:25 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi031 to 5697M 2024-04-25T00:25:26.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:25 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi031 2024-04-25T00:25:26.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:25 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi031 to 5697M 2024-04-25T00:25:26.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:26.294+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fca28062730 con 0x7fca140a2c20 2024-04-25T00:25:27.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:26 smithi169 ceph-mon[33112]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:27.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:26 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:26 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:27.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:26 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:27.143 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:26 smithi040 ceph-mon[32945]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:27.143 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:26 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:27.143 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:26 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:27.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:26 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:27.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:26 smithi031 ceph-mon[29167]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:27.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:27.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:27.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:26 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:28.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:27 smithi040 ceph-mon[32945]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:28.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:27 smithi031 ceph-mon[29167]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:28.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:27 smithi169 ceph-mon[33112]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:29.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:28 smithi040 ceph-mon[32945]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:29.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:28 smithi031 ceph-mon[29167]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:28 smithi169 ceph-mon[33112]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:29 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/788502597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]: dispatch 2024-04-25T00:25:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:29 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]: dispatch 2024-04-25T00:25:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:29 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]': finished 2024-04-25T00:25:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:29 smithi040 ceph-mon[32945]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:25:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:30.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:29 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/2571601159' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:29 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/788502597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]: dispatch 2024-04-25T00:25:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:29 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]: dispatch 2024-04-25T00:25:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:29 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]': finished 2024-04-25T00:25:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:29 smithi031 ceph-mon[29167]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:25:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:29 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/2571601159' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:30.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:29 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/788502597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]: dispatch 2024-04-25T00:25:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:29 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]: dispatch 2024-04-25T00:25:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:29 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc3fde10-5c71-4d9b-a4b0-b184780d5286"}]': finished 2024-04-25T00:25:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:29 smithi169 ceph-mon[33112]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:25:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:29 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/2571601159' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:31.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:31 smithi169 ceph-mon[33112]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:31 smithi040 ceph-mon[32945]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:31 smithi031 ceph-mon[29167]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:33.341 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:33 smithi040 ceph-mon[32945]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:33.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:33 smithi169 ceph-mon[33112]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:33 smithi031 ceph-mon[29167]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:35.259 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:35 smithi040 ceph-mon[32945]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:35.324 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:35 smithi031 ceph-mon[29167]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:35.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:35 smithi169 ceph-mon[33112]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:37.300 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:37 smithi040 ceph-mon[32945]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:37.301 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:37 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:25:37.301 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:37 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:37 smithi169 ceph-mon[33112]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:37.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:37 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:25:37.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:37 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:37 smithi031 ceph-mon[29167]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:37 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:25:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:37 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:38.273 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:38 smithi040 ceph-mon[32945]: Deploying daemon osd.7 on smithi040 2024-04-25T00:25:38.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:38 smithi169 ceph-mon[33112]: Deploying daemon osd.7 on smithi040 2024-04-25T00:25:38.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:38 smithi031 ceph-mon[29167]: Deploying daemon osd.7 on smithi040 2024-04-25T00:25:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:39 smithi169 ceph-mon[33112]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:39 smithi040 ceph-mon[32945]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:39.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:39 smithi031 ceph-mon[29167]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:41.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:41 smithi169 ceph-mon[33112]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:41 smithi040 ceph-mon[32945]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:41 smithi031 ceph-mon[29167]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:43.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:43 smithi169 ceph-mon[33112]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:43.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:43.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:43.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:43 smithi040 ceph-mon[32945]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:43 smithi031 ceph-mon[29167]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:45.334 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:45 smithi040 ceph-mon[32945]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:45.334 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.334 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.335 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:45.335 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:45.335 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:45 smithi169 ceph-mon[33112]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:45.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:45.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:45.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:45 smithi031 ceph-mon[29167]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:45.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.711+0000 7fca1e7fc700 1 -- 172.21.15.40:0/1435818119 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fca28062730 con 0x7fca140a2c20 2024-04-25T00:25:45.713 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 7 on host 'smithi040' 2024-04-25T00:25:45.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fca140a2c20 msgr2=0x7fca140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fca140a2c20 0x7fca140a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b5c0 tx=0x7fca18003680 comp rx=0 tx=0).stop 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 msgr2=0x7fca2819f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 0x7fca2819f970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca20007ec0 tx=0x7fca2000d3b0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 shutdown_connections 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fca140a2c20 0x7fca140a50e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28102d40 0x7fca281983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca28107fa0 0x7fca2819f970 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 --2- 172.21.15.40:0/1435818119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca28105670 0x7fca281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 >> 172.21.15.40:0/1435818119 conn(0x7fca280fc750 msgr2=0x7fca28103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 shutdown_connections 2024-04-25T00:25:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:25:45.714+0000 7fca2f343700 1 -- 172.21.15.40:0/1435818119 wait complete. 2024-04-25T00:25:46.406 DEBUG:teuthology.orchestra.run.smithi040:osd.7> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T00:25:46.409 INFO:tasks.cephadm:Deploying osd.8 on smithi169 with /dev/vg_nvme/lv_2... 2024-04-25T00:25:46.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:25:46.657 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:25:46.780 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:46 smithi169 ceph-mon[33112]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:46.780 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:46.780 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:46.780 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:46.780 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:46 smithi169 ceph-mon[33112]: from='osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:25:46.780 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:46 smithi169 ceph-mon[33112]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:25:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:46 smithi031 ceph-mon[29167]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:46 smithi031 ceph-mon[29167]: from='osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:25:47.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:46 smithi031 ceph-mon[29167]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:25:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:46 smithi040 ceph-mon[32945]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:47.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:46 smithi040 ceph-mon[32945]: from='osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:25:47.146 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:46 smithi040 ceph-mon[32945]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:25:47.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:47 smithi040 ceph-mon[32945]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:25:47.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:47 smithi040 ceph-mon[32945]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:25:47.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:47.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:47 smithi040 ceph-mon[32945]: from='osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:25:47.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:47 smithi040 ceph-mon[32945]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:25:47.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:47 smithi169 ceph-mon[33112]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:25:47.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:47 smithi169 ceph-mon[33112]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:25:47.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:47.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:47 smithi169 ceph-mon[33112]: from='osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:25:47.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:47 smithi169 ceph-mon[33112]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:25:48.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:47 smithi031 ceph-mon[29167]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:25:48.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:47 smithi031 ceph-mon[29167]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:25:48.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:47 smithi031 ceph-mon[29167]: from='osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:25:48.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:47 smithi031 ceph-mon[29167]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:25:48.395 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:25:48.169+0000 7fe81d4bd700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:25:48.396 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:25:48.182+0000 7fe814aa7700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:25:48.622 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:25:48.622 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:25:48.622 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T00:25:48.622 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T00:25:48.622 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115307 s, 909 MB/s 2024-04-25T00:25:48.622 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:48 smithi169 ceph-mon[33112]: from='osd.7 ' entity='osd.7' 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:48 smithi040 ceph-mon[32945]: from='osd.7 ' entity='osd.7' 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:48 smithi031 ceph-mon[29167]: from='osd.7 ' entity='osd.7' 2024-04-25T00:25:49.288 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-04-25T00:25:49.532 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:25:49.843 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] boot 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:49.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:49.845 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:25:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:25:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] boot 2024-04-25T00:25:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:25:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:49.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:25:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:25:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: osd.7 [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] boot 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:50.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 -- 172.21.15.169:0/2313475164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 msgr2=0x7f52441027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 --2- 172.21.15.169:0/2313475164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f52441027e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5238009990 tx=0x7f5238009ca0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 -- 172.21.15.169:0/2313475164 shutdown_connections 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 --2- 172.21.15.169:0/2313475164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5244102d20 0x7f52441095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 --2- 172.21.15.169:0/2313475164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5244101180 0x7f52441015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 --2- 172.21.15.169:0/2313475164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f52441027e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 -- 172.21.15.169:0/2313475164 >> 172.21.15.169:0/2313475164 conn(0x7f52440fc720 msgr2=0x7f52440feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 -- 172.21.15.169:0/2313475164 shutdown_connections 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 -- 172.21.15.169:0/2313475164 wait complete. 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.090+0000 7f5249a78700 1 Processor -- start 2024-04-25T00:25:51.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 -- start start 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5244101180 0x7f52441050c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f5244103760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5244102d20 0x7f5244103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f524410ccf0 con 0x7f5244102d20 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f524410cb70 con 0x7f5244101180 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5249a78700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f524410ce70 con 0x7f5244102380 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f5244103760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f5244103760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53840/0 (socket says 172.21.15.169:53840) 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 -- 172.21.15.169:0/371980017 learned_addr learned my addr 172.21.15.169:0/371980017 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f52437fe700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5244101180 0x7f52441050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5243fff700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5244102d20 0x7f5244103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 -- 172.21.15.169:0/371980017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5244101180 msgr2=0x7f52441050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5244101180 0x7f52441050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 -- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5244102d20 msgr2=0x7f5244103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5244102d20 0x7f5244103ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 -- 172.21.15.169:0/371980017 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5244104570 con 0x7f5244102380 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.091+0000 7f5242ffd700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f5244103760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f52380037f0 tx=0x7f52380047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.092+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238026070 con 0x7f5244102380 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.092+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5238004080 con 0x7f5244102380 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.092+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441a25a0 con 0x7f5244102380 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.092+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523800fe10 con 0x7f5244102380 2024-04-25T00:25:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.092+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a2b20 con 0x7f5244102380 2024-04-25T00:25:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.093+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5244068f20 con 0x7f5244102380 2024-04-25T00:25:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.094+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5238029070 con 0x7f5244102380 2024-04-25T00:25:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.094+0000 7f5240ff9700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f522c0a2ca0 0x7f522c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.095+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7f52380cf180 con 0x7f5244102380 2024-04-25T00:25:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.095+0000 7f52437fe700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f522c0a2ca0 0x7f522c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.095+0000 7f52437fe700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f522c0a2ca0 0x7f522c0a5160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52441021e0 tx=0x7f523400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.098 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.098+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5238030080 con 0x7f5244102380 2024-04-25T00:25:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:51 smithi169 ceph-mon[33112]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:51 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi040 2024-04-25T00:25:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:51 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi040 to 5697M 2024-04-25T00:25:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:51 smithi169 ceph-mon[33112]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:25:51.307 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:25:51.306+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f524406ae90 con 0x7f522c0a2ca0 2024-04-25T00:25:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:51 smithi040 ceph-mon[32945]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:51 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi040 2024-04-25T00:25:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:51 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi040 to 5697M 2024-04-25T00:25:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:51 smithi040 ceph-mon[32945]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:25:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:51 smithi031 ceph-mon[29167]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:25:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:51 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi040 2024-04-25T00:25:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:51 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi040 to 5697M 2024-04-25T00:25:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:51 smithi031 ceph-mon[29167]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:25:52.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:52 smithi169 ceph-mon[33112]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:25:52.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:52.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:52.263 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:52 smithi040 ceph-mon[32945]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:25:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:52 smithi031 ceph-mon[29167]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:25:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:25:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:25:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:53.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:53 smithi169 ceph-mon[33112]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:53.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:53 smithi169 ceph-mon[33112]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:53.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:53 smithi169 ceph-mon[33112]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T00:25:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:53 smithi040 ceph-mon[32945]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:53 smithi040 ceph-mon[32945]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:53 smithi040 ceph-mon[32945]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T00:25:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:53 smithi031 ceph-mon[29167]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:53 smithi031 ceph-mon[29167]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:25:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:53 smithi031 ceph-mon[29167]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T00:25:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:54 smithi169 ceph-mon[33112]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T00:25:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/1272698290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]: dispatch 2024-04-25T00:25:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:54 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]: dispatch 2024-04-25T00:25:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:54 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]': finished 2024-04-25T00:25:54.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:54 smithi169 ceph-mon[33112]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:25:54.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:25:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:54 smithi040 ceph-mon[32945]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T00:25:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/1272698290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]: dispatch 2024-04-25T00:25:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:54 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]: dispatch 2024-04-25T00:25:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:54 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]': finished 2024-04-25T00:25:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:54 smithi040 ceph-mon[32945]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:25:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:25:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:54 smithi031 ceph-mon[29167]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T00:25:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/1272698290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]: dispatch 2024-04-25T00:25:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:54 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]: dispatch 2024-04-25T00:25:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:54 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddad000a-a81c-4a3d-b5ef-ca98e6a57bab"}]': finished 2024-04-25T00:25:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:54 smithi031 ceph-mon[29167]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:25:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:25:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:55 smithi169 ceph-mon[33112]: pgmap v187: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:55 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/1624615833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:55.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:55 smithi040 ceph-mon[32945]: pgmap v187: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:55 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/1624615833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:55 smithi031 ceph-mon[29167]: pgmap v187: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:55 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/1624615833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:25:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:57 smithi169 ceph-mon[33112]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:57.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:57 smithi040 ceph-mon[32945]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:57 smithi031 ceph-mon[29167]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:59.247 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:25:59 smithi169 ceph-mon[33112]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:25:59 smithi040 ceph-mon[32945]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:25:59.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:25:59 smithi031 ceph-mon[29167]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:01.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:01 smithi169 ceph-mon[33112]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:01.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:01 smithi040 ceph-mon[32945]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:01.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:01 smithi031 ceph-mon[29167]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:02.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:02 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:26:02.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:02 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:02 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:26:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:02 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:02.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:02 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:26:02.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:02 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:03.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:03 smithi169 ceph-mon[33112]: Deploying daemon osd.8 on smithi169 2024-04-25T00:26:03.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:03 smithi169 ceph-mon[33112]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:03 smithi040 ceph-mon[32945]: Deploying daemon osd.8 on smithi169 2024-04-25T00:26:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:03 smithi040 ceph-mon[32945]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:03.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:03 smithi031 ceph-mon[29167]: Deploying daemon osd.8 on smithi169 2024-04-25T00:26:03.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:03 smithi031 ceph-mon[29167]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:05.206 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:05 smithi169 ceph-mon[33112]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:05 smithi040 ceph-mon[32945]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:05.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:05 smithi031 ceph-mon[29167]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:07.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:07 smithi169 ceph-mon[33112]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:07.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:07.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:07.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:07 smithi040 ceph-mon[32945]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:07 smithi031 ceph-mon[29167]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:09.309 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:09 smithi169 ceph-mon[33112]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:09 smithi040 ceph-mon[32945]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:09.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:09 smithi031 ceph-mon[29167]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:10.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.876+0000 7f5240ff9700 1 -- 172.21.15.169:0/371980017 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f524406ae90 con 0x7f522c0a2ca0 2024-04-25T00:26:10.877 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 8 on host 'smithi169' 2024-04-25T00:26:10.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.878+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f522c0a2ca0 msgr2=0x7f522c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.878+0000 7f5249a78700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f522c0a2ca0 0x7f522c0a5160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52441021e0 tx=0x7f523400b410 comp rx=0 tx=0).stop 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 msgr2=0x7f5244103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f5244103760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f52380037f0 tx=0x7f52380047b0 comp rx=0 tx=0).stop 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 shutdown_connections 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f522c0a2ca0 0x7f522c0a5160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5244102d20 0x7f5244103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5244101180 0x7f52441050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 --2- 172.21.15.169:0/371980017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5244102380 0x7f5244103760 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 >> 172.21.15.169:0/371980017 conn(0x7f52440fc720 msgr2=0x7f524406a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 shutdown_connections 2024-04-25T00:26:10.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:26:10.879+0000 7f5249a78700 1 -- 172.21.15.169:0/371980017 wait complete. 2024-04-25T00:26:11.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:11.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:26:11.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:26:11.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:26:11.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:26:11.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:11.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:11.539 DEBUG:teuthology.orchestra.run.smithi169:osd.8> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T00:26:11.541 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-04-25T00:26:11.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:26:11.801 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:26:12.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:12 smithi031 ceph-mon[29167]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:26:12.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:12 smithi031 ceph-mon[29167]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:26:12.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:12.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:12 smithi031 ceph-mon[29167]: from='osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:26:12.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:12 smithi031 ceph-mon[29167]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:26:12.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-mon[33112]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:26:12.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-mon[33112]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:26:12.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:12.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-mon[33112]: from='osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:26:12.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-mon[33112]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:26:12.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:26:12.145+0000 7f0244f33700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T00:26:12.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:26:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:26:12.154+0000 7f0240d26700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:26:12.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:12 smithi040 ceph-mon[32945]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:26:12.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:12 smithi040 ceph-mon[32945]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:26:12.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:12.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:12 smithi040 ceph-mon[32945]: from='osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:26:12.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:12 smithi040 ceph-mon[32945]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:26:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] boot 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:26:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] boot 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:26:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: osd.8 [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] boot 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:26:13.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:26:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:26:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:26:14.069 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T00:26:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T00:26:14.069 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112082 s, 936 MB/s 2024-04-25T00:26:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T00:26:14.363 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:14 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:26:14.364 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:14 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:26:14.364 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:14 smithi169 ceph-mon[33112]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:26:14.384 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:14 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:26:14.384 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:14 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:26:14.384 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:14 smithi031 ceph-mon[29167]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:26:14.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:14 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:26:14.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:14 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:26:14.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:14 smithi040 ceph-mon[32945]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:26:15.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-04-25T00:26:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:15 smithi031 ceph-mon[29167]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:26:15.292 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:15 smithi169 ceph-mon[33112]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:15.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:15.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:15 smithi040 ceph-mon[32945]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:26:16.453 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:16 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi169 2024-04-25T00:26:16.453 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:16 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T00:26:16.453 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:16 smithi031 ceph-mon[29167]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:26:16.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:16 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi169 2024-04-25T00:26:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:16 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T00:26:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:16 smithi169 ceph-mon[33112]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:26:16.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:16 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi169 2024-04-25T00:26:16.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:16 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T00:26:16.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:16 smithi040 ceph-mon[32945]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 -- 172.21.15.31:0/3278953293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8105640 msgr2=0x7f9ad8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 --2- 172.21.15.31:0/3278953293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8105640 0x7f9ad8107a30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009990 tx=0x7f9ac8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 -- 172.21.15.31:0/3278953293 shutdown_connections 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 --2- 172.21.15.31:0/3278953293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ad8107f70 0x7f9ad810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 --2- 172.21.15.31:0/3278953293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad8102d10 0x7f9ad8105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 --2- 172.21.15.31:0/3278953293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8105640 0x7f9ad8107a30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 -- 172.21.15.31:0/3278953293 >> 172.21.15.31:0/3278953293 conn(0x7f9ad80fc720 msgr2=0x7f9ad80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 -- 172.21.15.31:0/3278953293 shutdown_connections 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.122+0000 7f9adfadb700 1 -- 172.21.15.31:0/3278953293 wait complete. 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 Processor -- start 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 -- start start 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ad8102d10 0x7f9ad81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad8105640 0x7f9ad8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 0x7f9ad819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810dd80 con 0x7f9ad8107f70 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810dc00 con 0x7f9ad8102d10 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9adfadb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810df00 con 0x7f9ad8105640 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9ade078700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 0x7f9ad819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9ade078700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 0x7f9ad819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43576/0 (socket says 172.21.15.31:43576) 2024-04-25T00:26:17.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9ade078700 1 -- 172.21.15.31:0/2475864445 learned_addr learned my addr 172.21.15.31:0/2475864445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:26:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.123+0000 7f9add877700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ad8102d10 0x7f9ad81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9add076700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad8105640 0x7f9ad8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9ade078700 1 -- 172.21.15.31:0/2475864445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad8105640 msgr2=0x7f9ad8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9ade078700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad8105640 0x7f9ad8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9ade078700 1 -- 172.21.15.31:0/2475864445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ad8102d10 msgr2=0x7f9ad81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9ade078700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ad8102d10 0x7f9ad81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9ade078700 1 -- 172.21.15.31:0/2475864445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad81a0020 con 0x7f9ad8107f70 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9ade078700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 0x7f9ad819f980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0007d70 tx=0x7f9ad000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0016020 con 0x7f9ad8107f70 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad81a0310 con 0x7f9ad8107f70 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ad000ad00 con 0x7f9ad8107f70 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81a0850 con 0x7f9ad8107f70 2024-04-25T00:26:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.124+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00115e0 con 0x7f9ad8107f70 2024-04-25T00:26:17.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.125+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc005360 con 0x7f9ad8107f70 2024-04-25T00:26:17.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.126+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ad0019020 con 0x7f9ad8107f70 2024-04-25T00:26:17.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.127+0000 7f9aceffd700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ac40a2ca0 0x7f9ac40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.127+0000 7f9add877700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ac40a2ca0 0x7f9ac40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.127+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7f9ad00ce900 con 0x7f9ad8107f70 2024-04-25T00:26:17.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.127+0000 7f9add877700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ac40a2ca0 0x7f9ac40a5160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009a10 tx=0x7f9ad4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.131+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ad0020020 con 0x7f9ad8107f70 2024-04-25T00:26:17.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:17 smithi031 ceph-mon[29167]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:17.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:17 smithi031 ceph-mon[29167]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T00:26:17.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:17.347+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9abc000bf0 con 0x7f9ac40a2ca0 2024-04-25T00:26:17.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:17 smithi169 ceph-mon[33112]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:17.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:17 smithi169 ceph-mon[33112]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T00:26:17.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:17 smithi040 ceph-mon[32945]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:17.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:17 smithi040 ceph-mon[32945]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T00:26:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:18 smithi031 ceph-mon[29167]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:18 smithi031 ceph-mon[29167]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:26:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:26:18.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:18.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:18 smithi169 ceph-mon[33112]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:18 smithi169 ceph-mon[33112]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:26:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:26:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:18 smithi040 ceph-mon[32945]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:18 smithi040 ceph-mon[32945]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:26:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:26:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:20.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:20 smithi169 ceph-mon[33112]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:20 smithi040 ceph-mon[32945]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:20 smithi031 ceph-mon[29167]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/216890999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d854d1bd-461f-4def-b383-20908701640b"}]: dispatch 2024-04-25T00:26:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/216890999' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d854d1bd-461f-4def-b383-20908701640b"}]': finished 2024-04-25T00:26:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:21 smithi169 ceph-mon[33112]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:26:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2283173403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:26:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/216890999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d854d1bd-461f-4def-b383-20908701640b"}]: dispatch 2024-04-25T00:26:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/216890999' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d854d1bd-461f-4def-b383-20908701640b"}]': finished 2024-04-25T00:26:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:21 smithi040 ceph-mon[32945]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:26:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2283173403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:26:21.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/216890999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d854d1bd-461f-4def-b383-20908701640b"}]: dispatch 2024-04-25T00:26:21.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/216890999' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d854d1bd-461f-4def-b383-20908701640b"}]': finished 2024-04-25T00:26:21.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:21 smithi031 ceph-mon[29167]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:26:21.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:21.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2283173403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:26:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:22 smithi169 ceph-mon[33112]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:26:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:22 smithi040 ceph-mon[32945]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:26:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:22 smithi031 ceph-mon[29167]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:26:24.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:24 smithi169 ceph-mon[33112]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:26:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:24 smithi040 ceph-mon[32945]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:26:25.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:24 smithi031 ceph-mon[29167]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:26:26.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:26 smithi031 ceph-mon[29167]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:26:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:26 smithi169 ceph-mon[33112]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:26:26.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:26 smithi040 ceph-mon[32945]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:26:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:28 smithi169 ceph-mon[33112]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:26:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:28 smithi040 ceph-mon[32945]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:26:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:28 smithi031 ceph-mon[29167]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:26:29.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:26:29.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:29 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:29.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:29 smithi169 ceph-mon[33112]: Deploying daemon osd.9 on smithi031 2024-04-25T00:26:29.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:26:29.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:29 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:29.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:29 smithi040 ceph-mon[32945]: Deploying daemon osd.9 on smithi031 2024-04-25T00:26:30.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:26:30.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:29 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:30.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:29 smithi031 ceph-mon[29167]: Deploying daemon osd.9 on smithi031 2024-04-25T00:26:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:30 smithi169 ceph-mon[33112]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:26:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:30 smithi040 ceph-mon[32945]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:26:30.921 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:30 smithi031 ceph-mon[29167]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:26:32.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:32 smithi169 ceph-mon[33112]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:32 smithi040 ceph-mon[32945]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:32.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:32 smithi031 ceph-mon[29167]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:34 smithi031 ceph-mon[29167]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:34 smithi169 ceph-mon[33112]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:34 smithi040 ceph-mon[32945]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:35.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:35.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:35.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:36.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:36.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:36.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:36.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:36 smithi031 ceph-mon[29167]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:37.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:36 smithi169 ceph-mon[33112]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:37.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:36 smithi040 ceph-mon[32945]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:38.441 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-04-25T00:26:38.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.439+0000 7f9aceffd700 1 -- 172.21.15.31:0/2475864445 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9abc000bf0 con 0x7f9ac40a2ca0 2024-04-25T00:26:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ac40a2ca0 msgr2=0x7f9ac40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ac40a2ca0 0x7f9ac40a5160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009a10 tx=0x7f9ad4009190 comp rx=0 tx=0).stop 2024-04-25T00:26:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 msgr2=0x7f9ad819f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 0x7f9ad819f980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0007d70 tx=0x7f9ad000a340 comp rx=0 tx=0).stop 2024-04-25T00:26:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 shutdown_connections 2024-04-25T00:26:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ac40a2ca0 0x7f9ac40a5160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8107f70 0x7f9ad819f980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:38.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ad8102d10 0x7f9ad81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:38.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 --2- 172.21.15.31:0/2475864445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad8105640 0x7f9ad8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:38.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 >> 172.21.15.31:0/2475864445 conn(0x7f9ad80fc720 msgr2=0x7f9ad8103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:38.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 shutdown_connections 2024-04-25T00:26:38.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:26:38.441+0000 7f9adfadb700 1 -- 172.21.15.31:0/2475864445 wait complete. 2024-04-25T00:26:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:38 smithi031 ceph-mon[29167]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:38 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:38 smithi031 ceph-mon[29167]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:26:39.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:38 smithi169 ceph-mon[33112]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:39.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:38 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:39.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:38 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:39.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:38 smithi169 ceph-mon[33112]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:26:39.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:38 smithi040 ceph-mon[32945]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:39.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:38 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:39.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:38 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:39.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:38 smithi040 ceph-mon[32945]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:26:39.162 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T00:26:39.165 INFO:tasks.cephadm:Deploying osd.10 on smithi040 with /dev/vg_nvme/lv_1... 2024-04-25T00:26:39.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:26:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:26:40.687 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:40.688 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:26:40.688 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:26:40.688 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:26:40.688 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:40.688 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:40.688 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:40 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:40 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:41.227 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:26:40.873+0000 7efd50019700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T00:26:41.227 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:26:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:26:40.884+0000 7efd47e04700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:26:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:41 smithi031 ceph-mon[29167]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:26:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:41 smithi031 ceph-mon[29167]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T00:26:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:41 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:41 smithi031 ceph-mon[29167]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' 2024-04-25T00:26:41.593 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:26:41.593 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:26:41.593 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T00:26:41.593 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T00:26:41.593 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127612 s, 822 MB/s 2024-04-25T00:26:41.593 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T00:26:41.679 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:41 smithi040 ceph-mon[32945]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:26:41.679 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:41 smithi040 ceph-mon[32945]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T00:26:41.679 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:41.679 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:41 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:41.679 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:41 smithi040 ceph-mon[32945]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' 2024-04-25T00:26:41.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:41 smithi169 ceph-mon[33112]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T00:26:41.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:41 smithi169 ceph-mon[33112]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T00:26:41.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:41.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:41 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:41.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:41 smithi169 ceph-mon[33112]: from='osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503]' entity='osd.9' 2024-04-25T00:26:42.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_1 2024-04-25T00:26:42.450 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:26:42.676 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:26:42.677 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:26:42.677 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:42.677 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:42.677 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] boot 2024-04-25T00:26:42.677 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:26:42.677 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:42 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:42.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:26:42.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:26:42.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:42.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:42.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] boot 2024-04-25T00:26:42.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:26:42.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:42 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:26:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:26:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:26:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: osd.9 [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] boot 2024-04-25T00:26:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:26:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:42 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 -- 172.21.15.40:0/1047522677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 msgr2=0x7f57e4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/1047522677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4107a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009990 tx=0x7f57d4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 -- 172.21.15.40:0/1047522677 shutdown_connections 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/1047522677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e4102d20 0x7f57e4105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/1047522677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e4107f80 0x7f57e410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/1047522677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4107a40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.072+0000 7f57ec6fa700 1 -- 172.21.15.40:0/1047522677 >> 172.21.15.40:0/1047522677 conn(0x7f57e40fc730 msgr2=0x7f57e40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.073+0000 7f57ec6fa700 1 -- 172.21.15.40:0/1047522677 shutdown_connections 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.073+0000 7f57ec6fa700 1 -- 172.21.15.40:0/1047522677 wait complete. 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.073+0000 7f57ec6fa700 1 Processor -- start 2024-04-25T00:26:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.073+0000 7f57ec6fa700 1 -- start start 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ec6fa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e4102d20 0x7f57e41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ec6fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ec6fa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e4107f80 0x7f57e419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ec6fa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f57e410dd90 con 0x7f57e4107f80 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ec6fa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57e410dc10 con 0x7f57e4105650 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ec6fa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57e410df10 con 0x7f57e4102d20 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51628/0 (socket says 172.21.15.40:51628) 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 -- 172.21.15.40:0/2850906510 learned_addr learned my addr 172.21.15.40:0/2850906510 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57eac97700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e4107f80 0x7f57e419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57ea496700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e4102d20 0x7f57e41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 -- 172.21.15.40:0/2850906510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e4102d20 msgr2=0x7f57e41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e4102d20 0x7f57e41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 -- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e4107f80 msgr2=0x7f57e419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e4107f80 0x7f57e419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 -- 172.21.15.40:0/2850906510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e41a0040 con 0x7f57e4105650 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57e9c95700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4198920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f57d40098e0 tx=0x7f57d40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.074+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4027070 con 0x7f57e4105650 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.075+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57d4004110 con 0x7f57e4105650 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.075+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e41a02d0 con 0x7f57e4105650 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.075+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400f460 con 0x7f57e4105650 2024-04-25T00:26:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.075+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e41a0720 con 0x7f57e4105650 2024-04-25T00:26:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.076+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e4068f20 con 0x7f57e4105650 2024-04-25T00:26:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.077+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f57d402a070 con 0x7f57e4105650 2024-04-25T00:26:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.077+0000 7f57db7fe700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f57d00a2c20 0x7f57d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.077+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f57d40cf7e0 con 0x7f57e4105650 2024-04-25T00:26:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.078+0000 7f57ea496700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f57d00a2c20 0x7f57d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.078+0000 7f57ea496700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f57d00a2c20 0x7f57d00a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57e0005ea0 tx=0x7f57e0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:44.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.082+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f57d405e740 con 0x7f57e4105650 2024-04-25T00:26:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:26:44.301+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f57e4062670 con 0x7f57d00a2c20 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi031 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T00:26:44.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:44 smithi169 ceph-mon[33112]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi031 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:44.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:44.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:44.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:44 smithi040 ceph-mon[32945]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T00:26:44.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi031 2024-04-25T00:26:44.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:44.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:26:44.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:44 smithi031 ceph-mon[29167]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T00:26:45.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:45 smithi040 ceph-mon[32945]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:45.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:26:45.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:26:45.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:45 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:45 smithi031 ceph-mon[29167]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:26:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:26:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:45 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:45.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:45 smithi169 ceph-mon[33112]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:45.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:26:45.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:26:45.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:45 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:46.251 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:46 smithi040 ceph-mon[32945]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:46.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:46 smithi031 ceph-mon[29167]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:46.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:46 smithi169 ceph-mon[33112]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:47.480 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:47 smithi040 ceph-mon[32945]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:47.481 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:47 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/2669599815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]: dispatch 2024-04-25T00:26:47.481 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:47 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]: dispatch 2024-04-25T00:26:47.481 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:47 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]': finished 2024-04-25T00:26:47.481 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:47 smithi040 ceph-mon[32945]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:26:47.481 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:26:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:47 smithi031 ceph-mon[29167]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:47 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/2669599815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]: dispatch 2024-04-25T00:26:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:47 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]: dispatch 2024-04-25T00:26:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:47 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]': finished 2024-04-25T00:26:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:47 smithi031 ceph-mon[29167]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:26:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:26:47.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:47 smithi169 ceph-mon[33112]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:47.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:47 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/2669599815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]: dispatch 2024-04-25T00:26:47.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:47 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]: dispatch 2024-04-25T00:26:47.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:47 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "413c2a5c-f9a1-4a20-838d-624a8afae359"}]': finished 2024-04-25T00:26:47.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:47 smithi169 ceph-mon[33112]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:26:47.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:26:48.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:48 smithi031 ceph-mon[29167]: from='client.? 172.21.15.40:0/2984323163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:26:48.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:48 smithi169 ceph-mon[33112]: from='client.? 172.21.15.40:0/2984323163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:26:48.644 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:48 smithi040 ceph-mon[32945]: from='client.? 172.21.15.40:0/2984323163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:26:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:49 smithi031 ceph-mon[29167]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:49.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:49 smithi169 ceph-mon[33112]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:49.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:49 smithi040 ceph-mon[32945]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:50.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:50 smithi031 ceph-mon[29167]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:50.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:50 smithi169 ceph-mon[33112]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:50.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:50 smithi040 ceph-mon[32945]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:52.496 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:52 smithi040 ceph-mon[32945]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:52.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:52 smithi169 ceph-mon[33112]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:52 smithi031 ceph-mon[29167]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:54 smithi169 ceph-mon[33112]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:54 smithi040 ceph-mon[32945]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:54 smithi031 ceph-mon[29167]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:55.612 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:26:55.612 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:55.612 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:55 smithi040 ceph-mon[32945]: Deploying daemon osd.10 on smithi040 2024-04-25T00:26:55.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:26:55.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:55.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:55 smithi169 ceph-mon[33112]: Deploying daemon osd.10 on smithi040 2024-04-25T00:26:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:26:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:55 smithi031 ceph-mon[29167]: Deploying daemon osd.10 on smithi040 2024-04-25T00:26:56.641 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:56 smithi169 ceph-mon[33112]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:56.642 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:56 smithi040 ceph-mon[32945]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:56.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:56 smithi031 ceph-mon[29167]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:58.473 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:26:58 smithi040 ceph-mon[32945]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:58.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:26:58 smithi169 ceph-mon[33112]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:26:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:26:58 smithi031 ceph-mon[29167]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:00.388 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:00 smithi040 ceph-mon[32945]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:00.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:00 smithi169 ceph-mon[33112]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:00.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:00 smithi031 ceph-mon[29167]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:01.887 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:01.887 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:01.887 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:01.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:01.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:01.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:02.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:02.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:02.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:02.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:02 smithi169 ceph-mon[33112]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:02 smithi040 ceph-mon[32945]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:03.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:02 smithi031 ceph-mon[29167]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:04.574 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 10 on host 'smithi040' 2024-04-25T00:27:04.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.572+0000 7f57db7fe700 1 -- 172.21.15.40:0/2850906510 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f57e4062670 con 0x7f57d00a2c20 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f57d00a2c20 msgr2=0x7f57d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f57d00a2c20 0x7f57d00a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57e0005ea0 tx=0x7f57e0005e30 comp rx=0 tx=0).stop 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 msgr2=0x7f57e4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4198920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f57d40098e0 tx=0x7f57d40047b0 comp rx=0 tx=0).stop 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 shutdown_connections 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f57d00a2c20 0x7f57d00a50e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e4107f80 0x7f57e419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e4102d20 0x7f57e41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 --2- 172.21.15.40:0/2850906510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e4105650 0x7f57e4198920 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 >> 172.21.15.40:0/2850906510 conn(0x7f57e40fc730 msgr2=0x7f57e4103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.574+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 shutdown_connections 2024-04-25T00:27:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T00:27:04.575+0000 7f57ec6fa700 1 -- 172.21.15.40:0/2850906510 wait complete. 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:27:04.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:04 smithi169 ceph-mon[33112]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:27:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:04 smithi040 ceph-mon[32945]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:27:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:04 smithi031 ceph-mon[29167]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:27:05.375 DEBUG:teuthology.orchestra.run.smithi040:osd.10> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T00:27:05.378 INFO:tasks.cephadm:Deploying osd.11 on smithi169 with /dev/vg_nvme/lv_1... 2024-04-25T00:27:05.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:27:05.624 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:27:05.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-mon[32945]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:27:05.625 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:27:05.561+0000 7f8832686700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T00:27:05.625 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:27:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:27:05.577+0000 7f882e479700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:27:05.630 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:05 smithi169 ceph-mon[33112]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:27:05.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:05.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:27:05.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:27:05.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:05.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:05.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:27:05.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:05 smithi031 ceph-mon[29167]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T00:27:06.810 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:06 smithi040 ceph-mon[32945]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:06.811 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:06 smithi040 ceph-mon[32945]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:27:06.811 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:06 smithi040 ceph-mon[32945]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T00:27:06.811 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:06 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:06.811 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:06 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:06.878 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:06 smithi169 ceph-mon[33112]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:06.879 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:06 smithi169 ceph-mon[33112]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:27:06.879 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:06 smithi169 ceph-mon[33112]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T00:27:06.879 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:06 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:06.879 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:06 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:06 smithi031 ceph-mon[29167]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:06 smithi031 ceph-mon[29167]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T00:27:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:06 smithi031 ceph-mon[29167]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T00:27:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:06 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:06 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:27:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:27:07.750 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T00:27:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T00:27:07.750 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011653 s, 900 MB/s 2024-04-25T00:27:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T00:27:07.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:07 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:27:07.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:07 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:27:07.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:07.750 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:07 smithi169 ceph-mon[33112]: osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] boot 2024-04-25T00:27:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:07 smithi169 ceph-mon[33112]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:27:07.751 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:07 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:27:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:07 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:27:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:07 smithi040 ceph-mon[32945]: osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] boot 2024-04-25T00:27:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:07 smithi040 ceph-mon[32945]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:27:07.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:08.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:07 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:27:08.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:07 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:27:08.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:08.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:07 smithi031 ceph-mon[29167]: osd.10 [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] boot 2024-04-25T00:27:08.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:07 smithi031 ceph-mon[29167]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:27:08.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:27:08.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-04-25T00:27:08.720 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:08.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:08.753 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:27:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:08.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:08.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:08.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:08.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:27:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:27:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:09.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:09.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:09.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:09.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:09.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:09.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:09.837 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:09 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi040 2024-04-25T00:27:09.837 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:09 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi040 to 4273M 2024-04-25T00:27:09.837 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:09 smithi169 ceph-mon[33112]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:27:09.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:09 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi040 2024-04-25T00:27:09.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:09 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi040 to 4273M 2024-04-25T00:27:09.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:09 smithi040 ceph-mon[32945]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:27:10.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:09 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi040 2024-04-25T00:27:10.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:09 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi040 to 4273M 2024-04-25T00:27:10.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:09 smithi031 ceph-mon[29167]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:27:10.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/1368929973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 msgr2=0x7fbfe010a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:10.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/1368929973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe010a6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009990 tx=0x7fbfdc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:27:10.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/1368929973 shutdown_connections 2024-04-25T00:27:10.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/1368929973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe010a6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:10.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/1368929973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe0102290 0x7fbfe01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:10.355 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/1368929973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0103490 0x7fbfe01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/1368929973 >> 172.21.15.169:0/1368929973 conn(0x7fbfe00fd830 msgr2=0x7fbfe00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/1368929973 shutdown_connections 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/1368929973 wait complete. 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 Processor -- start 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.353+0000 7fbfe7b3c700 1 -- start start 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0102290 0x7fbfe0075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe0103490 0x7fbfe0074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe0074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe010dda0 con 0x7fbfe0102290 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe010dc20 con 0x7fbfe0103490 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe010df20 con 0x7fbfe0103e30 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe0074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe0074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38314/0 (socket says 172.21.15.169:38314) 2024-04-25T00:27:10.356 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 -- 172.21.15.169:0/2194440275 learned_addr learned my addr 172.21.15.169:0/2194440275 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe58d8700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0102290 0x7fbfe0075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe50d7700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe0103490 0x7fbfe0074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 -- 172.21.15.169:0/2194440275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe0103490 msgr2=0x7fbfe0074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe0103490 0x7fbfe0074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 -- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0102290 msgr2=0x7fbfe0075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0102290 0x7fbfe0075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 -- 172.21.15.169:0/2194440275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe0075410 con 0x7fbfe0103e30 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe58d8700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0102290 0x7fbfe0075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe60d9700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe0074b60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0059d0 tx=0x7fbfdc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc026070 con 0x7fbfe0103e30 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfdc005030 con 0x7fbfe0103e30 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe01a2480 con 0x7fbfe0103e30 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc004b90 con 0x7fbfe0103e30 2024-04-25T00:27:10.357 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.354+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe01a2950 con 0x7fbfe0103e30 2024-04-25T00:27:10.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.355+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe0068f20 con 0x7fbfe0103e30 2024-04-25T00:27:10.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.356+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbfdc029070 con 0x7fbfe0103e30 2024-04-25T00:27:10.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.357+0000 7fbfd6ffd700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbfcc0a2c20 0x7fbfcc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:10.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.357+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7fbfdc0cf680 con 0x7fbfe0103e30 2024-04-25T00:27:10.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.357+0000 7fbfe58d8700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbfcc0a2c20 0x7fbfcc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:10.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.358+0000 7fbfe58d8700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbfcc0a2c20 0x7fbfcc0a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfe01032f0 tx=0x7fbfd8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:10.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.362+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfdc05d740 con 0x7fbfe0103e30 2024-04-25T00:27:10.565 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:10.565+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbfe0062670 con 0x7fbfcc0a2c20 2024-04-25T00:27:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:10 smithi169 ceph-mon[33112]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:10 smithi169 ceph-mon[33112]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:27:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:27:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:27:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:10 smithi040 ceph-mon[32945]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:10 smithi040 ceph-mon[32945]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:27:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:27:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:27:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:11.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:10 smithi031 ceph-mon[29167]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:11.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:10 smithi031 ceph-mon[29167]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:27:11.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:27:11.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:27:11.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:11.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:11 smithi169 ceph-mon[33112]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:27:11.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:11 smithi169 ceph-mon[33112]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T00:27:11.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:11 smithi040 ceph-mon[32945]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:27:11.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:11 smithi040 ceph-mon[32945]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T00:27:12.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:11 smithi031 ceph-mon[29167]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:27:12.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:11 smithi031 ceph-mon[29167]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T00:27:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:12 smithi169 ceph-mon[33112]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:27:12.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:12 smithi040 ceph-mon[32945]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:27:13.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:12 smithi031 ceph-mon[29167]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:27:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:13 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/1903084259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]: dispatch 2024-04-25T00:27:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:13 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]: dispatch 2024-04-25T00:27:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:13 smithi169 ceph-mon[33112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]': finished 2024-04-25T00:27:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:13 smithi169 ceph-mon[33112]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:27:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:13 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/1903084259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]: dispatch 2024-04-25T00:27:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:13 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]: dispatch 2024-04-25T00:27:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:13 smithi040 ceph-mon[32945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]': finished 2024-04-25T00:27:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:13 smithi040 ceph-mon[32945]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:27:13.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:13 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/1903084259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]: dispatch 2024-04-25T00:27:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:13 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]: dispatch 2024-04-25T00:27:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:13 smithi031 ceph-mon[29167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d81d667-30bf-47da-824f-51eb05444477"}]': finished 2024-04-25T00:27:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:13 smithi031 ceph-mon[29167]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:27:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:14.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:14 smithi169 ceph-mon[33112]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T00:27:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:14 smithi169 ceph-mon[33112]: from='client.? 172.21.15.169:0/3395540647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:27:14.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:14 smithi040 ceph-mon[32945]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T00:27:14.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:14 smithi040 ceph-mon[32945]: from='client.? 172.21.15.169:0/3395540647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:27:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:14 smithi031 ceph-mon[29167]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T00:27:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:14 smithi031 ceph-mon[29167]: from='client.? 172.21.15.169:0/3395540647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:27:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:16 smithi169 ceph-mon[33112]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:27:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:16 smithi040 ceph-mon[32945]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:27:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:16 smithi031 ceph-mon[29167]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:27:18.771 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:18 smithi169 ceph-mon[33112]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T00:27:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:18 smithi040 ceph-mon[32945]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T00:27:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:18 smithi031 ceph-mon[29167]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T00:27:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:20 smithi169 ceph-mon[33112]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:20 smithi040 ceph-mon[32945]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:20 smithi031 ceph-mon[29167]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:21.869 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:27:21.869 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:21.869 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:21 smithi169 ceph-mon[33112]: Deploying daemon osd.11 on smithi169 2024-04-25T00:27:21.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:27:21.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:21.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:21 smithi040 ceph-mon[32945]: Deploying daemon osd.11 on smithi169 2024-04-25T00:27:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:27:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:22.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:21 smithi031 ceph-mon[29167]: Deploying daemon osd.11 on smithi169 2024-04-25T00:27:22.876 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:22 smithi169 ceph-mon[33112]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:22 smithi040 ceph-mon[32945]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:22 smithi031 ceph-mon[29167]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:24.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:24 smithi169 ceph-mon[33112]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:24.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:24 smithi040 ceph-mon[32945]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:25.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:24 smithi031 ceph-mon[29167]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:26.809 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:26 smithi169 ceph-mon[33112]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:26 smithi040 ceph-mon[32945]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:27.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:26 smithi031 ceph-mon[29167]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:28.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:28.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:28.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:28 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:28.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:28.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:28.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:28 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:28.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:28.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:28.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:28 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:29 smithi169 ceph-mon[33112]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:29.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:29 smithi040 ceph-mon[32945]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:29 smithi031 ceph-mon[29167]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:30.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.535+0000 7fbfd6ffd700 1 -- 172.21.15.169:0/2194440275 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbfe0062670 con 0x7fbfcc0a2c20 2024-04-25T00:27:30.537 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 11 on host 'smithi169' 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbfcc0a2c20 msgr2=0x7fbfcc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbfcc0a2c20 0x7fbfcc0a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfe01032f0 tx=0x7fbfd8009570 comp rx=0 tx=0).stop 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 msgr2=0x7fbfe0074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe0074b60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0059d0 tx=0x7fbfdc009f90 comp rx=0 tx=0).stop 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 shutdown_connections 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbfcc0a2c20 0x7fbfcc0a50e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfe0102290 0x7fbfe0075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe0103e30 0x7fbfe0074b60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 --2- 172.21.15.169:0/2194440275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe0103490 0x7fbfe0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 >> 172.21.15.169:0/2194440275 conn(0x7fbfe00fd830 msgr2=0x7fbfe01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 shutdown_connections 2024-04-25T00:27:30.539 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:30.538+0000 7fbfe7b3c700 1 -- 172.21.15.169:0/2194440275 wait complete. 2024-04-25T00:27:31.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:31 smithi169 ceph-mon[33112]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:31.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:31.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:31 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:31.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:31 smithi169 ceph-mon[33112]: from='osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:27:31.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:31 smithi169 ceph-mon[33112]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:27:31.228 DEBUG:teuthology.orchestra.run.smithi169:osd.11> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T00:27:31.230 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T00:27:31.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T00:27:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:31 smithi040 ceph-mon[32945]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:31 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:31 smithi040 ceph-mon[32945]: from='osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:27:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:31 smithi040 ceph-mon[32945]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:27:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:27:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:31 smithi031 ceph-mon[29167]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:31 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:31 smithi031 ceph-mon[29167]: from='osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:27:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:31 smithi031 ceph-mon[29167]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:27:32.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:32.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:32.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:32.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:27:32.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: from='osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:27:32.528 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:27:32.529 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:32.529 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-mon[33112]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:27:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:27:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: from='osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:27:32.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:27:32.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:32.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:32 smithi031 ceph-mon[29167]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:27:32.889 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:27:32.527+0000 7f5d8a248700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T00:27:32.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:27:32 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:27:32.544+0000 7f5d8603b700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: from='osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:32 smithi040 ceph-mon[32945]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 -- 172.21.15.31:0/2441455746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103e30 msgr2=0x7f8f7c10a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/2441455746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103e30 0x7f8f7c10a6e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400b0d0 tx=0x7f8f7400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 -- 172.21.15.31:0/2441455746 shutdown_connections 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/2441455746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103e30 0x7f8f7c10a6e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/2441455746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c102290 0x7f8f7c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/2441455746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c103490 0x7f8f7c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 -- 172.21.15.31:0/2441455746 >> 172.21.15.31:0/2441455746 conn(0x7f8f7c0fd830 msgr2=0x7f8f7c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 -- 172.21.15.31:0/2441455746 shutdown_connections 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.478+0000 7f8f80f4c700 1 -- 172.21.15.31:0/2441455746 wait complete. 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 Processor -- start 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 -- start start 2024-04-25T00:27:33.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c102290 0x7f8f7c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 0x7f8f7c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c103e30 0x7f8f7c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10dd80 con 0x7f8f7c103490 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10dc00 con 0x7f8f7c103e30 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f80f4c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10df00 con 0x7f8f7c102290 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f79d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 0x7f8f7c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f79d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 0x7f8f7c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55820/0 (socket says 172.21.15.31:55820) 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f79d9b700 1 -- 172.21.15.31:0/530783598 learned_addr learned my addr 172.21.15.31:0/530783598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.479+0000 7f8f7ad9d700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c103e30 0x7f8f7c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f7a59c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c102290 0x7f8f7c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f79d9b700 1 -- 172.21.15.31:0/530783598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c102290 msgr2=0x7f8f7c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f79d9b700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c102290 0x7f8f7c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f79d9b700 1 -- 172.21.15.31:0/530783598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c103e30 msgr2=0x7f8f7c19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f79d9b700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c103e30 0x7f8f7c19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f79d9b700 1 -- 172.21.15.31:0/530783598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c19fff0 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f7a59c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c102290 0x7f8f7c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f79d9b700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 0x7f8f7c1988d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b560 tx=0x7f8f6400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64015020 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f64011ce0 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1a02e0 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64010d60 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.480+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c1a0850 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.481+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c068f20 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.482+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f64018020 con 0x7f8f7c103490 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.483+0000 7f8f737fe700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2c20 0x7f8f680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.483+0000 7f8f7a59c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2c20 0x7f8f680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.483+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7f8f640ce360 con 0x7f8f7c103490 2024-04-25T00:27:33.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.483+0000 7f8f7a59c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2c20 0x7f8f680a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c005fd0 tx=0x7f8f6c005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:33.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.488+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f6401f020 con 0x7f8f7c103490 2024-04-25T00:27:33.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:33 smithi169 ceph-mon[33112]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:27:33.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:33 smithi169 ceph-mon[33112]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T00:27:33.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:33.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:33 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.701+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8f7c062e40 con 0x7f8f7c103490 2024-04-25T00:27:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.701+0000 7f8f737fe700 1 -- 172.21.15.31:0/530783598 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f8f6409b7e0 con 0x7f8f7c103490 2024-04-25T00:27:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2c20 msgr2=0x7f8f680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2c20 0x7f8f680a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c005fd0 tx=0x7f8f6c005f00 comp rx=0 tx=0).stop 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 msgr2=0x7f8f7c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 0x7f8f7c1988d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b560 tx=0x7f8f6400b870 comp rx=0 tx=0).stop 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 shutdown_connections 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c103e30 0x7f8f7c19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c102290 0x7f8f7c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2c20 0x7f8f680a50e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 --2- 172.21.15.31:0/530783598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c103490 0x7f8f7c1988d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.703+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 >> 172.21.15.31:0/530783598 conn(0x7f8f7c0fd830 msgr2=0x7f8f7c106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.704+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 shutdown_connections 2024-04-25T00:27:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:33.704+0000 7f8f80f4c700 1 -- 172.21.15.31:0/530783598 wait complete. 2024-04-25T00:27:33.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:33 smithi031 ceph-mon[29167]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:27:33.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:33 smithi031 ceph-mon[29167]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T00:27:33.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:33.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:33 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:33 smithi040 ceph-mon[32945]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T00:27:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:33 smithi040 ceph-mon[32945]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T00:27:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:33 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:34.413 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714004853,"num_in_osds":12,"osd_in_since":1714004833,"num_remapped_pgs":0} 2024-04-25T00:27:34.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:27:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:27:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: purged_snaps scrub starts 2024-04-25T00:27:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: purged_snaps scrub ok 2024-04-25T00:27:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] boot 2024-04-25T00:27:34.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:27:34.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:34.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/530783598' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:27:34.835 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: purged_snaps scrub starts 2024-04-25T00:27:34.835 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: purged_snaps scrub ok 2024-04-25T00:27:34.836 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:34.836 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] boot 2024-04-25T00:27:34.836 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:27:34.836 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:34.836 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/530783598' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: purged_snaps scrub starts 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: purged_snaps scrub ok 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: osd.11 [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] boot 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:27:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/530783598' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:27:35.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:27:35.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: Detected new or changed devices on smithi169 2024-04-25T00:27:35.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:35.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:35 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: Detected new or changed devices on smithi169 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T00:27:35.886 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:35.887 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:35 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: Detected new or changed devices on smithi169 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T00:27:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:35.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:35 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 -- 172.21.15.31:0/383311323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8006a8e0 msgr2=0x7f3f8010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 --2- 172.21.15.31:0/383311323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8006a8e0 0x7f3f8010f0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b0a0 tx=0x7f3f7800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 -- 172.21.15.31:0/383311323 shutdown_connections 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 --2- 172.21.15.31:0/383311323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8006a8e0 0x7f3f8010f0f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 --2- 172.21.15.31:0/383311323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f801082b0 0x7f3f80108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 --2- 172.21.15.31:0/383311323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f80069f40 0x7f3f8006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 -- 172.21.15.31:0/383311323 >> 172.21.15.31:0/383311323 conn(0x7f3f800780e0 msgr2=0x7f3f800784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 -- 172.21.15.31:0/383311323 shutdown_connections 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.369+0000 7f3f86f23700 1 -- 172.21.15.31:0/383311323 wait complete. 2024-04-25T00:27:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 Processor -- start 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 -- start start 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f80069f40 0x7f3f8019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8006a8e0 0x7f3f8019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 0x7f3f80198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801123e0 con 0x7f3f801082b0 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80112260 con 0x7f3f8006a8e0 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f86f23700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80112560 con 0x7f3f80069f40 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f854c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 0x7f3f80198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f854c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 0x7f3f80198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55844/0 (socket says 172.21.15.31:55844) 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f854c0700 1 -- 172.21.15.31:0/1918884581 learned_addr learned my addr 172.21.15.31:0/1918884581 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f84cbf700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f80069f40 0x7f3f8019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.370+0000 7f3f7ffff700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8006a8e0 0x7f3f8019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f854c0700 1 -- 172.21.15.31:0/1918884581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f80069f40 msgr2=0x7f3f8019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f854c0700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f80069f40 0x7f3f8019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f854c0700 1 -- 172.21.15.31:0/1918884581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8006a8e0 msgr2=0x7f3f8019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f854c0700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8006a8e0 0x7f3f8019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f854c0700 1 -- 172.21.15.31:0/1918884581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f80198d80 con 0x7f3f801082b0 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f854c0700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 0x7f3f80198620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b830 tx=0x7f3f78004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78021070 con 0x7f3f801082b0 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f80199010 con 0x7f3f801082b0 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f78007e30 con 0x7f3f801082b0 2024-04-25T00:27:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78004700 con 0x7f3f801082b0 2024-04-25T00:27:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.371+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801a5330 con 0x7f3f801082b0 2024-04-25T00:27:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.372+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f80068f20 con 0x7f3f801082b0 2024-04-25T00:27:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.373+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3f78007980 con 0x7f3f801082b0 2024-04-25T00:27:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.374+0000 7f3f7dffb700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3f680a2d20 0x7f3f680a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.374+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f3f7800f020 con 0x7f3f801082b0 2024-04-25T00:27:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.374+0000 7f3f84cbf700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3f680a2d20 0x7f3f680a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.374+0000 7f3f84cbf700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3f680a2d20 0x7f3f680a51e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000abd0 tx=0x7f3f7000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.379+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f7802a020 con 0x7f3f801082b0 2024-04-25T00:27:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.584+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f8002db30 con 0x7f3f801082b0 2024-04-25T00:27:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.585+0000 7f3f7dffb700 1 -- 172.21.15.31:0/1918884581 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18446 (secure 0 0 0) 0x7f3f7809d470 con 0x7f3f801082b0 2024-04-25T00:27:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:36.586 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:27:34.530272+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:27:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3f680a2d20 msgr2=0x7f3f680a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3f680a2d20 0x7f3f680a51e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000abd0 tx=0x7f3f7000a380 comp rx=0 tx=0).stop 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 msgr2=0x7f3f80198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 0x7f3f80198620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b830 tx=0x7f3f78004ce0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 shutdown_connections 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3f680a2d20 0x7f3f680a51e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f801082b0 0x7f3f80198620 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f80069f40 0x7f3f8019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 --2- 172.21.15.31:0/1918884581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8006a8e0 0x7f3f8019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 >> 172.21.15.31:0/1918884581 conn(0x7f3f800780e0 msgr2=0x7f3f800ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 shutdown_connections 2024-04-25T00:27:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:36.588+0000 7f3f86f23700 1 -- 172.21.15.31:0/1918884581 wait complete. 2024-04-25T00:27:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:37 smithi031 ceph-mon[29167]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:37 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1918884581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:37.388 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:27:37.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T00:27:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:37 smithi169 ceph-mon[33112]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:37 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1918884581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:37 smithi040 ceph-mon[32945]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:37 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1918884581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:27:39.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:39 smithi031 ceph-mon[29167]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:39 smithi169 ceph-mon[33112]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:39 smithi040 ceph-mon[32945]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.455+0000 7f8fc22e6700 1 -- 172.21.15.31:0/1606110214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc06ac40 msgr2=0x7f8fbc06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.455+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/1606110214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc06b020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0099c0 tx=0x7f8fac009cd0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.455+0000 7f8fc22e6700 1 -- 172.21.15.31:0/1606110214 shutdown_connections 2024-04-25T00:27:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.455+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/1606110214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc06b020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/1606110214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc107590 0x7f8fbc10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/1606110214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 0x7f8fbc106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 -- 172.21.15.31:0/1606110214 >> 172.21.15.31:0/1606110214 conn(0x7f8fbc0781d0 msgr2=0x7f8fbc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 -- 172.21.15.31:0/1606110214 shutdown_connections 2024-04-25T00:27:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 -- 172.21.15.31:0/1606110214 wait complete. 2024-04-25T00:27:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 Processor -- start 2024-04-25T00:27:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.456+0000 7f8fc22e6700 1 -- start start 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc22e6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc22e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 0x7f8fbc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc22e6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 0x7f8fbc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc22e6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc112420 con 0x7f8fbc107590 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc22e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1122a0 con 0x7f8fbc06b5f0 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc22e6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1125a0 con 0x7f8fbc06ac40 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 0x7f8fbc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 0x7f8fbc1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55858/0 (socket says 172.21.15.31:55858) 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 -- 172.21.15.31:0/373985459 learned_addr learned my addr 172.21.15.31:0/373985459 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fbbfff700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fbb7fe700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 0x7f8fbc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 -- 172.21.15.31:0/373985459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc06ac40 msgr2=0x7f8fbc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 -- 172.21.15.31:0/373985459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 msgr2=0x7f8fbc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 0x7f8fbc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fc0883700 1 -- 172.21.15.31:0/373985459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc198fc0 con 0x7f8fbc107590 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fbbfff700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.457+0000 7f8fbb7fe700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 0x7f8fbc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.458+0000 7f8fc0883700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 0x7f8fbc1986e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007ee0 tx=0x7f8fb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.458+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400dbd0 con 0x7f8fbc107590 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.458+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb400dd70 con 0x7f8fbc107590 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.458+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc1992b0 con 0x7f8fbc107590 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.458+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4005a20 con 0x7f8fbc107590 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.458+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1a5540 con 0x7f8fbc107590 2024-04-25T00:27:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.459+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc103990 con 0x7f8fbc107590 2024-04-25T00:27:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.461+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8fb401a020 con 0x7f8fbc107590 2024-04-25T00:27:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.461+0000 7f8fb97fa700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.461+0000 7f8fbbfff700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.461+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f8fb40cea00 con 0x7f8fbc107590 2024-04-25T00:27:39.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.462+0000 7f8fbbfff700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fac000c00 tx=0x7f8fac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.465+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8fb4010070 con 0x7f8fbc107590 2024-04-25T00:27:39.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.673+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8fbc1994e0 con 0x7f8fbc107590 2024-04-25T00:27:39.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.673+0000 7f8fb97fa700 1 -- 172.21.15.31:0/373985459 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f8fb409be00 con 0x7f8fbc107590 2024-04-25T00:27:39.675 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:27:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.675+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8fa40a2c20 msgr2=0x7f8fa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.675+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fac000c00 tx=0x7f8fac005fd0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.675+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 msgr2=0x7f8fbc1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.675+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 0x7f8fbc1986e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007ee0 tx=0x7f8fb400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 shutdown_connections 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc107590 0x7f8fbc1986e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc06ac40 0x7f8fbc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 --2- 172.21.15.31:0/373985459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fbc06b5f0 0x7f8fbc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 >> 172.21.15.31:0/373985459 conn(0x7f8fbc0781d0 msgr2=0x7f8fbc0ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 shutdown_connections 2024-04-25T00:27:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:39.676+0000 7f8fc22e6700 1 -- 172.21.15.31:0/373985459 wait complete. 2024-04-25T00:27:40.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:40 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/373985459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:40.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:40 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/373985459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:40.405 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:40 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/373985459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:40.436 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.b on smithi040 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.e on smithi040 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.h on smithi040 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.k on smithi040 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.c on smithi169 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.f on smithi169 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.i on smithi169 2024-04-25T00:27:40.437 INFO:tasks.cephadm:Adding mds.l on smithi169 2024-04-25T00:27:40.438 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l' 2024-04-25T00:27:40.689 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:41.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:41 smithi169 ceph-mon[33112]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:41 smithi040 ceph-mon[32945]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:41 smithi031 ceph-mon[29167]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:42.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 -- 172.21.15.169:0/3439141014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38101ec0 msgr2=0x7f0d38102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 --2- 172.21.15.169:0/3439141014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38101ec0 0x7f0d38102340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b0a0 tx=0x7f0d3000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 -- 172.21.15.169:0/3439141014 shutdown_connections 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 --2- 172.21.15.169:0/3439141014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38100ba0 0x7f0d38100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 --2- 172.21.15.169:0/3439141014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38101ec0 0x7f0d38102340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 --2- 172.21.15.169:0/3439141014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101500 0x7f0d38101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 -- 172.21.15.169:0/3439141014 >> 172.21.15.169:0/3439141014 conn(0x7f0d380fc760 msgr2=0x7f0d380feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 -- 172.21.15.169:0/3439141014 shutdown_connections 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.342+0000 7f0d3d695700 1 -- 172.21.15.169:0/3439141014 wait complete. 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 Processor -- start 2024-04-25T00:27:42.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 -- start start 2024-04-25T00:27:42.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 0x7f0d3819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:42.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38101500 0x7f0d3819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:42.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101ec0 0x7f0d381a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3810ff80 con 0x7f0d38101500 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3810fe00 con 0x7f0d38101ec0 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d3d695700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38110100 con 0x7f0d38100ba0 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 0x7f0d3819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 0x7f0d3819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59350/0 (socket says 172.21.15.169:59350) 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.343+0000 7f0d36ffd700 1 -- 172.21.15.169:0/4011366576 learned_addr learned my addr 172.21.15.169:0/4011366576 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d367fc700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38101500 0x7f0d3819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d377fe700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101ec0 0x7f0d381a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d36ffd700 1 -- 172.21.15.169:0/4011366576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101ec0 msgr2=0x7f0d381a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d36ffd700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101ec0 0x7f0d381a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d36ffd700 1 -- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38101500 msgr2=0x7f0d3819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d36ffd700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38101500 0x7f0d3819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d36ffd700 1 -- 172.21.15.169:0/4011366576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d381a22a0 con 0x7f0d38100ba0 2024-04-25T00:27:42.347 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d377fe700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101ec0 0x7f0d381a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d367fc700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38101500 0x7f0d3819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d36ffd700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 0x7f0d3819a640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d980 tx=0x7f0d2800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28016070 con 0x7f0d38100ba0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d381a2590 con 0x7f0d38100ba0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d28010460 con 0x7f0d38100ba0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800f620 con 0x7f0d38100ba0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.344+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381a2b50 con 0x7f0d38100ba0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.345+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d38068f20 con 0x7f0d38100ba0 2024-04-25T00:27:42.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.346+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d28019070 con 0x7f0d38100ba0 2024-04-25T00:27:42.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.347+0000 7f0d1ffff700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0d200a2c20 0x7f0d200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:42.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.347+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0d280ce7a0 con 0x7f0d38100ba0 2024-04-25T00:27:42.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.347+0000 7f0d367fc700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0d200a2c20 0x7f0d200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:42.350 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.348+0000 7f0d367fc700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0d200a2c20 0x7f0d200a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0099d0 tx=0x7f0d2c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:42.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.352+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d2809bba0 con 0x7f0d38100ba0 2024-04-25T00:27:42.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.557+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}) -- 0x7f0d38062930 con 0x7f0d200a2c20 2024-04-25T00:27:42.568 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.567+0000 7f0d1ffff700 1 -- 172.21.15.169:0/4011366576 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0d38062930 con 0x7f0d200a2c20 2024-04-25T00:27:42.568 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mds.all update... 2024-04-25T00:27:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0d200a2c20 msgr2=0x7f0d200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0d200a2c20 0x7f0d200a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0099d0 tx=0x7f0d2c008040 comp rx=0 tx=0).stop 2024-04-25T00:27:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 msgr2=0x7f0d3819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 0x7f0d3819a640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d980 tx=0x7f0d2800dd40 comp rx=0 tx=0).stop 2024-04-25T00:27:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 shutdown_connections 2024-04-25T00:27:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d38101ec0 0x7f0d381a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d38100ba0 0x7f0d3819a640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0d200a2c20 0x7f0d200a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 --2- 172.21.15.169:0/4011366576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d38101500 0x7f0d3819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:42.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 >> 172.21.15.169:0/4011366576 conn(0x7f0d380fc760 msgr2=0x7f0d380fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:42.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 shutdown_connections 2024-04-25T00:27:42.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T00:27:42.569+0000 7f0d3d695700 1 -- 172.21.15.169:0/4011366576 wait complete. 2024-04-25T00:27:43.236 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:43 smithi169 ceph-mon[33112]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:43.236 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:43.236 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:43.264 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T00:27:43.267 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T00:27:43.310 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T00:27:43.354 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T00:27:43.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:43 smithi031 ceph-mon[29167]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:43.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:43.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:43.374 DEBUG:teuthology.orchestra.run.smithi040:mds.b> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T00:27:43.377 DEBUG:teuthology.orchestra.run.smithi040:mds.e> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T00:27:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:43 smithi040 ceph-mon[32945]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:43.396 DEBUG:teuthology.orchestra.run.smithi040:mds.h> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T00:27:43.439 DEBUG:teuthology.orchestra.run.smithi040:mds.k> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T00:27:43.482 DEBUG:teuthology.orchestra.run.smithi169:mds.c> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T00:27:43.484 DEBUG:teuthology.orchestra.run.smithi169:mds.f> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T00:27:43.528 DEBUG:teuthology.orchestra.run.smithi169:mds.i> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T00:27:43.570 DEBUG:teuthology.orchestra.run.smithi169:mds.l> sudo journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T00:27:43.614 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T00:27:43.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.727+0000 7fdaff464640 1 -- 172.21.15.31:0/1057188894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf81006a0 msgr2=0x7fdaf8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.727+0000 7fdaff464640 1 --2- 172.21.15.31:0/1057188894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf81006a0 0x7fdaf8100ac0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009900 tx=0x7fdae802a410 comp rx=0 tx=0).stop 2024-04-25T00:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- 172.21.15.31:0/1057188894 shutdown_connections 2024-04-25T00:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 --2- 172.21.15.31:0/1057188894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdaf8102250 0x7fdaf8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 --2- 172.21.15.31:0/1057188894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf8101890 0x7fdaf8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 --2- 172.21.15.31:0/1057188894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf81006a0 0x7fdaf8100ac0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- 172.21.15.31:0/1057188894 >> 172.21.15.31:0/1057188894 conn(0x7fdaf80fbc30 msgr2=0x7fdaf80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- 172.21.15.31:0/1057188894 shutdown_connections 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- 172.21.15.31:0/1057188894 wait complete. 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 Processor -- start 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- start start 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf81006a0 0x7fdaf8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdaf8101890 0x7fdaf8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 0x7fdaf819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810c1a0 con 0x7fdaf8102250 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810c020 con 0x7fdaf8101890 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.728+0000 7fdaff464640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810c320 con 0x7fdaf81006a0 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 0x7fdaf819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 0x7fdaf819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34030/0 (socket says 172.21.15.31:34030) 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 -- 172.21.15.31:0/774533330 learned_addr learned my addr 172.21.15.31:0/774533330 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafd200640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf81006a0 0x7fdaf8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafc9ff640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdaf8101890 0x7fdaf8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 -- 172.21.15.31:0/774533330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf81006a0 msgr2=0x7fdaf8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf81006a0 0x7fdaf8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 -- 172.21.15.31:0/774533330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdaf8101890 msgr2=0x7fdaf8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdaf8101890 0x7fdaf8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 -- 172.21.15.31:0/774533330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf819f640 con 0x7fdaf8102250 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdafda01640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 0x7fdaf819efd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000eaa0 tx=0x7fdaf000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000cdb0 con 0x7fdaf8102250 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaf0005020 con 0x7fdaf8102250 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00116d0 con 0x7fdaf8102250 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf819f930 con 0x7fdaf8102250 2024-04-25T00:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.729+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf819fe80 con 0x7fdaf8102250 2024-04-25T00:27:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.730+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf81a0170 con 0x7fdaf8102250 2024-04-25T00:27:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.731+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdaf00051d0 con 0x7fdaf8102250 2024-04-25T00:27:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.732+0000 7fdaee7fc640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdae00a2c10 0x7fdae00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.732+0000 7fdafd200640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdae00a2c10 0x7fdae00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.732+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fdaf00cec30 con 0x7fdaf8102250 2024-04-25T00:27:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.733+0000 7fdafd200640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdae00a2c10 0x7fdae00a50d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004770 tx=0x7fdae8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.738+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdaf0016030 con 0x7fdaf8102250 2024-04-25T00:27:43.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.924+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaf81a0340 con 0x7fdaf8102250 2024-04-25T00:27:43.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.925+0000 7fdaee7fc640 1 -- 172.21.15.31:0/774533330 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18446 (secure 0 0 0) 0x7fdaf009b9e0 con 0x7fdaf8102250 2024-04-25T00:27:43.927 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:43.927 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:27:34.530272+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.927+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdae00a2c10 msgr2=0x7fdae00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.927+0000 7fdaff464640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdae00a2c10 0x7fdae00a50d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004770 tx=0x7fdae8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.927+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 msgr2=0x7fdaf819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.927+0000 7fdaff464640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 0x7fdaf819efd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000eaa0 tx=0x7fdaf000ef60 comp rx=0 tx=0).stop 2024-04-25T00:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 shutdown_connections 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdae00a2c10 0x7fdae00a50d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102250 0x7fdaf819efd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdaf8101890 0x7fdaf8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 --2- 172.21.15.31:0/774533330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf81006a0 0x7fdaf8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 >> 172.21.15.31:0/774533330 conn(0x7fdaf80fbc30 msgr2=0x7fdaf8069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 shutdown_connections 2024-04-25T00:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:43.928+0000 7fdaff464640 1 -- 172.21.15.31:0/774533330 wait complete. 2024-04-25T00:27:43.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:27:43.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:27:44.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 -- 172.21.15.31:0/1048585694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f80101890 msgr2=0x7f8f80101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1048585694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f80101890 0x7f8f80101d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009900 tx=0x7f8f7402a410 comp rx=0 tx=0).stop 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 -- 172.21.15.31:0/1048585694 shutdown_connections 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1048585694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f80102250 0x7f8f80108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1048585694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f80101890 0x7f8f80101d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1048585694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f801006a0 0x7f8f80100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 -- 172.21.15.31:0/1048585694 >> 172.21.15.31:0/1048585694 conn(0x7f8f800fbc10 msgr2=0x7f8f800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 -- 172.21.15.31:0/1048585694 shutdown_connections 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 -- 172.21.15.31:0/1048585694 wait complete. 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.044+0000 7f8f8655a640 1 Processor -- start 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 -- start start 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 0x7f8f8006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f80101890 0x7f8f8019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f80102250 0x7f8f8019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8010c1b0 con 0x7f8f801006a0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8010c030 con 0x7f8f80102250 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f8655a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8010c330 con 0x7f8f80101890 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 0x7f8f8006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 0x7f8f8006a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34056/0 (socket says 172.21.15.31:34056) 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 -- 172.21.15.31:0/1353646104 learned_addr learned my addr 172.21.15.31:0/1353646104 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7f7fe640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f80101890 0x7f8f8019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 -- 172.21.15.31:0/1353646104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f80101890 msgr2=0x7f8f8019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f80101890 0x7f8f8019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 -- 172.21.15.31:0/1353646104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f80102250 msgr2=0x7f8f8019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f80102250 0x7f8f8019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 -- 172.21.15.31:0/1353646104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8019f660 con 0x7f8f801006a0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.045+0000 7f8f7ffff640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 0x7f8f8006a8d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d950 tx=0x7f8f7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.047+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70017020 con 0x7f8f801006a0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.047+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8019f950 con 0x7f8f801006a0 2024-04-25T00:27:44.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.047+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f70009c30 con 0x7f8f801006a0 2024-04-25T00:27:44.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.047+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000f6b0 con 0x7f8f801006a0 2024-04-25T00:27:44.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.047+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8019fe70 con 0x7f8f801006a0 2024-04-25T00:27:44.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.047+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f80107ab0 con 0x7f8f801006a0 2024-04-25T00:27:44.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.048+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f7001a020 con 0x7f8f801006a0 2024-04-25T00:27:44.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.048+0000 7f8f7d7fa640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2b90 0x7f8f680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.048+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f8f700cebc0 con 0x7f8f801006a0 2024-04-25T00:27:44.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.048+0000 7f8f7f7fe640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2b90 0x7f8f680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.049+0000 7f8f7f7fe640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2b90 0x7f8f680a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006010 tx=0x7f8f74000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:44.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.052+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f70009de0 con 0x7f8f801006a0 2024-04-25T00:27:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.237+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f8002d320 con 0x7f8f801006a0 2024-04-25T00:27:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.238+0000 7f8f7d7fa640 1 -- 172.21.15.31:0/1353646104 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f8f7009bfc0 con 0x7f8f801006a0 2024-04-25T00:27:44.239 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:27:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2b90 msgr2=0x7f8f680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2b90 0x7f8f680a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006010 tx=0x7f8f74000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 msgr2=0x7f8f8006a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 0x7f8f8006a8d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d950 tx=0x7f8f7000de10 comp rx=0 tx=0).stop 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 shutdown_connections 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f80102250 0x7f8f8019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f80101890 0x7f8f8019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f680a2b90 0x7f8f680a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 --2- 172.21.15.31:0/1353646104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f801006a0 0x7f8f8006a8d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 >> 172.21.15.31:0/1353646104 conn(0x7f8f800fbc10 msgr2=0x7f8f80104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 shutdown_connections 2024-04-25T00:27:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.240+0000 7f8f8655a640 1 -- 172.21.15.31:0/1353646104 wait complete. 2024-04-25T00:27:44.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:44.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:44 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/774533330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:44.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:44.339 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:44.340 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:44 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/774533330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:44.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 -- 172.21.15.31:0/3152237555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c102980 msgr2=0x7f0b7c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 --2- 172.21.15.31:0/3152237555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c102980 0x7f0b7c104df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b7002a410 comp rx=0 tx=0).stop 2024-04-25T00:27:44.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 -- 172.21.15.31:0/3152237555 shutdown_connections 2024-04-25T00:27:44.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 --2- 172.21.15.31:0/3152237555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7c105330 0x7f0b7c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 --2- 172.21.15.31:0/3152237555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c102980 0x7f0b7c104df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 --2- 172.21.15.31:0/3152237555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b7c100010 0x7f0b7c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 -- 172.21.15.31:0/3152237555 >> 172.21.15.31:0/3152237555 conn(0x7f0b7c0fbc30 msgr2=0x7f0b7c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 -- 172.21.15.31:0/3152237555 shutdown_connections 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.353+0000 7f0b83f29640 1 -- 172.21.15.31:0/3152237555 wait complete. 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 Processor -- start 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 -- start start 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 0x7f0b7c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7c102980 0x7f0b7c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b7c105330 0x7f0b7c1a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1138c0 con 0x7f0b7c100010 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c113740 con 0x7f0b7c102980 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b83f29640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c113a40 con 0x7f0b7c105330 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b81cc5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 0x7f0b7c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b81cc5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 0x7f0b7c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34072/0 (socket says 172.21.15.31:34072) 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b81cc5640 1 -- 172.21.15.31:0/2248370156 learned_addr learned my addr 172.21.15.31:0/2248370156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b824c6640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b7c105330 0x7f0b7c1a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.354+0000 7f0b814c4640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7c102980 0x7f0b7c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b81cc5640 1 -- 172.21.15.31:0/2248370156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b7c105330 msgr2=0x7f0b7c1a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b81cc5640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b7c105330 0x7f0b7c1a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b81cc5640 1 -- 172.21.15.31:0/2248370156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7c102980 msgr2=0x7f0b7c19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b81cc5640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7c102980 0x7f0b7c19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b81cc5640 1 -- 172.21.15.31:0/2248370156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c1a5bb0 con 0x7f0b7c100010 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b81cc5640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 0x7f0b7c19df90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00ea80 tx=0x7f0b6c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00ce40 con 0x7f0b7c100010 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b6c005020 con 0x7f0b7c100010 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c1a5ea0 con 0x7f0b7c100010 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0116d0 con 0x7f0b7c100010 2024-04-25T00:27:44.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.355+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1a6300 con 0x7f0b7c100010 2024-04-25T00:27:44.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.356+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c0fd720 con 0x7f0b7c100010 2024-04-25T00:27:44.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.357+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0b6c0051d0 con 0x7f0b7c100010 2024-04-25T00:27:44.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.358+0000 7f0b7affd640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0b640a2c10 0x7f0b640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:44.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.358+0000 7f0b814c4640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0b640a2c10 0x7f0b640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:44.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.358+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0b6c0cede0 con 0x7f0b7c100010 2024-04-25T00:27:44.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.358+0000 7f0b814c4640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0b640a2c10 0x7f0b640a50d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0b70006010 tx=0x7f0b70000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:44.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.362+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0b6c016030 con 0x7f0b7c100010 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:44.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:44.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:44.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:44 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/774533330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:44.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.584+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b7c02d170 con 0x7f0b7c100010 2024-04-25T00:27:44.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.585+0000 7f0b7affd640 1 -- 172.21.15.31:0/2248370156 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f0b6c09baf0 con 0x7f0b7c100010 2024-04-25T00:27:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T00:27:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-04-25T00:27:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0b640a2c10 msgr2=0x7f0b640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0b640a2c10 0x7f0b640a50d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0b70006010 tx=0x7f0b70000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 msgr2=0x7f0b7c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 0x7f0b7c19df90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00ea80 tx=0x7f0b6c00ef40 comp rx=0 tx=0).stop 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 shutdown_connections 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b7c105330 0x7f0b7c1a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7c102980 0x7f0b7c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0b640a2c10 0x7f0b640a50d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 --2- 172.21.15.31:0/2248370156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b7c100010 0x7f0b7c19df90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 >> 172.21.15.31:0/2248370156 conn(0x7f0b7c0fbc30 msgr2=0x7f0b7c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 shutdown_connections 2024-04-25T00:27:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:44.587+0000 7f0b83f29640 1 -- 172.21.15.31:0/2248370156 wait complete. 2024-04-25T00:27:45.309 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:45 smithi169 ceph-mon[33112]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:45.309 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:45 smithi169 ceph-mon[33112]: Deploying daemon mds.l on smithi169 2024-04-25T00:27:45.309 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:45 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1353646104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:45.309 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:45 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2248370156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:45 smithi040 ceph-mon[32945]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:45 smithi040 ceph-mon[32945]: Deploying daemon mds.l on smithi169 2024-04-25T00:27:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:45 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1353646104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:45 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2248370156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:45 smithi031 ceph-mon[29167]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:45 smithi031 ceph-mon[29167]: Deploying daemon mds.l on smithi169 2024-04-25T00:27:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:45 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1353646104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:45.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:45 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2248370156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:45.640 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 systemd[1]: Starting Ceph mds.l for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:27:45.976 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 podman[63421]: 2024-04-25 00:27:45.612386474 +0000 UTC m=+0.018876343 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:27:45.976 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 podman[63421]: 2024-04-25 00:27:45.721874664 +0000 UTC m=+0.128364520 container create 2200eac577be8f9ab7c0bde2abb9fd559f78c5337d9666e5779e9be4fbb36690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T00:27:45.976 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 podman[63421]: 2024-04-25 00:27:45.949914594 +0000 UTC m=+0.356404450 container init 2200eac577be8f9ab7c0bde2abb9fd559f78c5337d9666e5779e9be4fbb36690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T00:27:45.976 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 podman[63421]: 2024-04-25 00:27:45.952559071 +0000 UTC m=+0.359048939 container start 2200eac577be8f9ab7c0bde2abb9fd559f78c5337d9666e5779e9be4fbb36690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:27:46.389 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 ceph-mds[63440]: -- 172.21.15.169:0/2360557228 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560aa6045a40 con 0x560aa6018c00 2024-04-25T00:27:46.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l[63436]: starting mds.l at 2024-04-25T00:27:46.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:46 smithi169 bash[63421]: 2200eac577be8f9ab7c0bde2abb9fd559f78c5337d9666e5779e9be4fbb36690 2024-04-25T00:27:46.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:27:46 smithi169 systemd[1]: Started Ceph mds.l for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:27:46.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 -- 172.21.15.31:0/1412400652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70101890 msgr2=0x7f9c70101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/1412400652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70101890 0x7f9c70101d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009900 tx=0x7f9c6002a410 comp rx=0 tx=0).stop 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 -- 172.21.15.31:0/1412400652 shutdown_connections 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/1412400652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c70102250 0x7f9c70108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/1412400652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70101890 0x7f9c70101d10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/1412400652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c701006a0 0x7f9c70100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 -- 172.21.15.31:0/1412400652 >> 172.21.15.31:0/1412400652 conn(0x7f9c700fbc30 msgr2=0x7f9c700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 -- 172.21.15.31:0/1412400652 shutdown_connections 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 -- 172.21.15.31:0/1412400652 wait complete. 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 Processor -- start 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.713+0000 7f9c77ffb640 1 -- start start 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c77ffb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c701006a0 0x7f9c70104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c77ffb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c70101890 0x7f9c70102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c77ffb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 0x7f9c701031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c77ffb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7010c220 con 0x7f9c70102250 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c77ffb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7010c0a0 con 0x7f9c701006a0 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c77ffb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7010c3a0 con 0x7f9c70101890 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 0x7f9c701031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 0x7f9c701031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34088/0 (socket says 172.21.15.31:34088) 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 -- 172.21.15.31:0/454674604 learned_addr learned my addr 172.21.15.31:0/454674604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c75596640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c70101890 0x7f9c70102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c75d97640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c701006a0 0x7f9c70104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 -- 172.21.15.31:0/454674604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c70101890 msgr2=0x7f9c70102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c70101890 0x7f9c70102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 -- 172.21.15.31:0/454674604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c701006a0 msgr2=0x7f9c70104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c701006a0 0x7f9c70104570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 -- 172.21.15.31:0/454674604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c70103980 con 0x7f9c70102250 2024-04-25T00:27:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c76598640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 0x7f9c701031b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800eb10 tx=0x7f9c6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.714+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68004b20 con 0x7f9c70102250 2024-04-25T00:27:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.715+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c68004cc0 con 0x7f9c70102250 2024-04-25T00:27:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.715+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68010690 con 0x7f9c70102250 2024-04-25T00:27:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.715+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c701a1b10 con 0x7f9c70102250 2024-04-25T00:27:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.715+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c701a2030 con 0x7f9c70102250 2024-04-25T00:27:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.716+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c701a2260 con 0x7f9c70102250 2024-04-25T00:27:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.717+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9c68005020 con 0x7f9c70102250 2024-04-25T00:27:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.717+0000 7f9c66ffd640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c5c0a2b90 0x7f9c5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.717+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f9c680cf1f0 con 0x7f9c70102250 2024-04-25T00:27:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.717+0000 7f9c75d97640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c5c0a2b90 0x7f9c5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.718+0000 7f9c75d97640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c5c0a2b90 0x7f9c5c0a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c005fd0 tx=0x7f9c6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:46.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.722+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c6809c5f0 con 0x7f9c70102250 2024-04-25T00:27:46.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.944+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c701a2430 con 0x7f9c70102250 2024-04-25T00:27:46.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.944+0000 7f9c66ffd640 1 -- 172.21.15.31:0/454674604 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f9c6809bef0 con 0x7f9c70102250 2024-04-25T00:27:46.945 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:46.945 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:27:46.945 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c5c0a2b90 msgr2=0x7f9c5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c5c0a2b90 0x7f9c5c0a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c005fd0 tx=0x7f9c6c00b410 comp rx=0 tx=0).stop 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 msgr2=0x7f9c701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 0x7f9c701031b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800eb10 tx=0x7f9c6800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 shutdown_connections 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c5c0a2b90 0x7f9c5c0a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70102250 0x7f9c701031b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c70101890 0x7f9c70102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 --2- 172.21.15.31:0/454674604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c701006a0 0x7f9c70104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:46.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 >> 172.21.15.31:0/454674604 conn(0x7f9c700fbc30 msgr2=0x7f9c700fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:46.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 shutdown_connections 2024-04-25T00:27:46.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:46.946+0000 7f9c77ffb640 1 -- 172.21.15.31:0/454674604 wait complete. 2024-04-25T00:27:47.093 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:47.093 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] up:boot 2024-04-25T00:27:47.093 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: fsmap 1 up:standby 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: Deploying daemon mds.k on smithi040 2024-04-25T00:27:47.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:46 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/454674604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] up:boot 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: fsmap 1 up:standby 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:47.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:47.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:47.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: Deploying daemon mds.k on smithi040 2024-04-25T00:27:47.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:46 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/454674604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] up:boot 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: fsmap 1 up:standby 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:47.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:47.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: Deploying daemon mds.k on smithi040 2024-04-25T00:27:47.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:46 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/454674604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:48.578 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 systemd[1]: Starting Ceph mds.k for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:27:48.828 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 podman[62181]: 2024-04-25 00:27:48.481203666 +0000 UTC m=+0.018478817 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:27:48.829 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 podman[62181]: 2024-04-25 00:27:48.592480621 +0000 UTC m=+0.129755757 container create 28e2f54d7edda45090007d67cf880778731c1d4b2badfdbea90c85bf05208859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:27:48.829 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 podman[62181]: 2024-04-25 00:27:48.82842427 +0000 UTC m=+0.365699406 container init 28e2f54d7edda45090007d67cf880778731c1d4b2badfdbea90c85bf05208859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:27:48.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 -- 172.21.15.31:0/4242773483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c101350 msgr2=0x7f758c1017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 --2- 172.21.15.31:0/4242773483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c101350 0x7f758c1017d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7584009ca0 tx=0x7f758402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 -- 172.21.15.31:0/4242773483 shutdown_connections 2024-04-25T00:27:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 --2- 172.21.15.31:0/4242773483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c101350 0x7f758c1017d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 --2- 172.21.15.31:0/4242773483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f758c1009b0 0x7f758c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 --2- 172.21.15.31:0/4242773483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 0x7f758c100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 -- 172.21.15.31:0/4242773483 >> 172.21.15.31:0/4242773483 conn(0x7f758c0fbbf0 msgr2=0x7f758c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 -- 172.21.15.31:0/4242773483 shutdown_connections 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.084+0000 7f7592abd640 1 -- 172.21.15.31:0/4242773483 wait complete. 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 Processor -- start 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 -- start start 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 0x7f758c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 0x7f758c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f758c101350 0x7f758c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10f480 con 0x7f758c1009b0 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10f300 con 0x7f758c100050 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7592abd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10f600 con 0x7f758c101350 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f758bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 0x7f758c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f758bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 0x7f758c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34116/0 (socket says 172.21.15.31:34116) 2024-04-25T00:27:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f758bfff640 1 -- 172.21.15.31:0/1241064642 learned_addr learned my addr 172.21.15.31:0/1241064642 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.085+0000 7f7590859640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 0x7f758c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f759105a640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f758c101350 0x7f758c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f758bfff640 1 -- 172.21.15.31:0/1241064642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f758c101350 msgr2=0x7f758c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f758bfff640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f758c101350 0x7f758c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f758bfff640 1 -- 172.21.15.31:0/1241064642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 msgr2=0x7f758c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f758bfff640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 0x7f758c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f758bfff640 1 -- 172.21.15.31:0/1241064642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c1a1750 con 0x7f758c1009b0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f7590859640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 0x7f758c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f758bfff640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 0x7f758c19a070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f757800d950 tx=0x7f757800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578017030 con 0x7f758c1009b0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1a1a40 con 0x7f758c1009b0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7578004020 con 0x7f758c1009b0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75780047a0 con 0x7f758c1009b0 2024-04-25T00:27:49.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.086+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1a1fe0 con 0x7f758c1009b0 2024-04-25T00:27:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.087+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c10af80 con 0x7f758c1009b0 2024-04-25T00:27:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.089+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75780041d0 con 0x7f758c1009b0 2024-04-25T00:27:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.089+0000 7f7589ffb640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f757c0a2b90 0x7f757c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.089+0000 7f7590859640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f757c0a2b90 0x7f757c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.089+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f75780ce5e0 con 0x7f758c1009b0 2024-04-25T00:27:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.090+0000 7f7590859640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f757c0a2b90 0x7f757c0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fc0 tx=0x7f7580005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.094+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f757809b940 con 0x7f758c1009b0 2024-04-25T00:27:49.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:48 smithi040 ceph-mon[32945]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:49.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:49.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:49.145 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 podman[62181]: 2024-04-25 00:27:48.831325734 +0000 UTC m=+0.368600870 container start 28e2f54d7edda45090007d67cf880778731c1d4b2badfdbea90c85bf05208859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:27:49.145 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 ceph-mds[62200]: -- 172.21.15.40:0/1673096862 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f116da7a40 con 0x55f116d7a800 2024-04-25T00:27:49.145 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k[62196]: starting mds.k at 2024-04-25T00:27:49.146 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 bash[62181]: 28e2f54d7edda45090007d67cf880778731c1d4b2badfdbea90c85bf05208859 2024-04-25T00:27:49.146 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:27:48 smithi040 systemd[1]: Started Ceph mds.k for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:27:49.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:48 smithi031 ceph-mon[29167]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:49.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:49.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.335+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f758c062340 con 0x7f758c1009b0 2024-04-25T00:27:49.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.336+0000 7f7589ffb640 1 -- 172.21.15.31:0/1241064642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f757809b240 con 0x7f758c1009b0 2024-04-25T00:27:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:49.336 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:27:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f757c0a2b90 msgr2=0x7f757c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f757c0a2b90 0x7f757c0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fc0 tx=0x7f7580005f90 comp rx=0 tx=0).stop 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 msgr2=0x7f758c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 0x7f758c19a070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f757800d950 tx=0x7f757800de10 comp rx=0 tx=0).stop 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 shutdown_connections 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f758c101350 0x7f758c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f757c0a2b90 0x7f757c0a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c1009b0 0x7f758c19a070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 --2- 172.21.15.31:0/1241064642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f758c100050 0x7f758c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 >> 172.21.15.31:0/1241064642 conn(0x7f758c0fbbf0 msgr2=0x7f758c0fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 shutdown_connections 2024-04-25T00:27:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:49.338+0000 7f7592abd640 1 -- 172.21.15.31:0/1241064642 wait complete. 2024-04-25T00:27:49.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:48 smithi169 ceph-mon[33112]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:50.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6832/3504624607,v1:172.21.15.40:6833/3504624607] up:boot 2024-04-25T00:27:50.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: fsmap 2 up:standby 2024-04-25T00:27:50.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:27:50.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:50.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:50.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:50.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:50.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: Deploying daemon mds.j on smithi031 2024-04-25T00:27:50.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:49 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1241064642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:50.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6832/3504624607,v1:172.21.15.40:6833/3504624607] up:boot 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: fsmap 2 up:standby 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: Deploying daemon mds.j on smithi031 2024-04-25T00:27:50.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:49 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1241064642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6832/3504624607,v1:172.21.15.40:6833/3504624607] up:boot 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: fsmap 2 up:standby 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:50.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: Deploying daemon mds.j on smithi031 2024-04-25T00:27:50.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:49 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1241064642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:51.259 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:51 smithi040 ceph-mon[32945]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:51.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:51 smithi031 ceph-mon[29167]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:51.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:51.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:51 smithi169 ceph-mon[33112]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 -- 172.21.15.31:0/3836024740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 msgr2=0x7f28c01045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 --2- 172.21.15.31:0/3836024740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c01045f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009900 tx=0x7f28b002a410 comp rx=0 tx=0).stop 2024-04-25T00:27:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 -- 172.21.15.31:0/3836024740 shutdown_connections 2024-04-25T00:27:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 --2- 172.21.15.31:0/3836024740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28c0107460 0x7f28c01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 --2- 172.21.15.31:0/3836024740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0104b30 0x7f28c0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 --2- 172.21.15.31:0/3836024740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c01045f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 -- 172.21.15.31:0/3836024740 >> 172.21.15.31:0/3836024740 conn(0x7f28c00fbbf0 msgr2=0x7f28c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 -- 172.21.15.31:0/3836024740 shutdown_connections 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.465+0000 7f28c8441640 1 -- 172.21.15.31:0/3836024740 wait complete. 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 Processor -- start 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 -- start start 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28c0104b30 0x7f28c0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 0x7f28c019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28c010d2b0 con 0x7f28c0107460 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28c010d130 con 0x7f28c0102200 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c8441640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28c010d430 con 0x7f28c0104b30 2024-04-25T00:27:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c69de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 0x7f28c019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c61dd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c69de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 0x7f28c019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34132/0 (socket says 172.21.15.31:34132) 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.466+0000 7f28c69de640 1 -- 172.21.15.31:0/1178012253 learned_addr learned my addr 172.21.15.31:0/1178012253 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c59dc640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28c0104b30 0x7f28c0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c69de640 1 -- 172.21.15.31:0/1178012253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28c0104b30 msgr2=0x7f28c0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c69de640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28c0104b30 0x7f28c0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c69de640 1 -- 172.21.15.31:0/1178012253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 msgr2=0x7f28c0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c69de640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c69de640 1 -- 172.21.15.31:0/1178012253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c019f640 con 0x7f28c0107460 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c61dd640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c69de640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 0x7f28c019efd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009fd0 tx=0x7f28b800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8017070 con 0x7f28c0107460 2024-04-25T00:27:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28b8003e20 con 0x7f28c0107460 2024-04-25T00:27:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c019f930 con 0x7f28c0107460 2024-04-25T00:27:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8010460 con 0x7f28c0107460 2024-04-25T00:27:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.467+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c019fe30 con 0x7f28c0107460 2024-04-25T00:27:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.468+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c00fd6e0 con 0x7f28c0107460 2024-04-25T00:27:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.469+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f28b8003970 con 0x7f28c0107460 2024-04-25T00:27:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.470+0000 7f28b77fe640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f28ac0a2b90 0x7f28ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.470+0000 7f28c61dd640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f28ac0a2b90 0x7f28ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.470+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f28b80ce940 con 0x7f28c0107460 2024-04-25T00:27:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.471+0000 7f28c61dd640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f28ac0a2b90 0x7f28ac0a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006010 tx=0x7f28b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.475+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f28b809b6f0 con 0x7f28c0107460 2024-04-25T00:27:51.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.701+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c002d170 con 0x7f28c0107460 2024-04-25T00:27:51.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.701+0000 7f28b77fe640 1 -- 172.21.15.31:0/1178012253 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f28b809f9f0 con 0x7f28c0107460 2024-04-25T00:27:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:51.702 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:27:51.702 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2024-04-25T00:27:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f28ac0a2b90 msgr2=0x7f28ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f28ac0a2b90 0x7f28ac0a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006010 tx=0x7f28b0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 msgr2=0x7f28c019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 0x7f28c019efd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009fd0 tx=0x7f28b800b260 comp rx=0 tx=0).stop 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 shutdown_connections 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f28ac0a2b90 0x7f28ac0a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c0107460 0x7f28c019efd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28c0104b30 0x7f28c0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 --2- 172.21.15.31:0/1178012253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28c0102200 0x7f28c0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 >> 172.21.15.31:0/1178012253 conn(0x7f28c00fbbf0 msgr2=0x7f28c0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 shutdown_connections 2024-04-25T00:27:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:51.704+0000 7f28c8441640 1 -- 172.21.15.31:0/1178012253 wait complete. 2024-04-25T00:27:52.022 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:27:51 smithi031 systemd[1]: Started Ceph mds.j for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1178012253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1178012253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1178012253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:52.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:53 smithi031 ceph-mon[29167]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:53 smithi031 ceph-mon[29167]: Deploying daemon mds.d on smithi031 2024-04-25T00:27:53.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:53 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] up:boot 2024-04-25T00:27:53.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:53 smithi031 ceph-mon[29167]: fsmap 3 up:standby 2024-04-25T00:27:53.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:27:53.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:53 smithi169 ceph-mon[33112]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:53.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:53 smithi169 ceph-mon[33112]: Deploying daemon mds.d on smithi031 2024-04-25T00:27:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:53 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] up:boot 2024-04-25T00:27:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:53 smithi169 ceph-mon[33112]: fsmap 3 up:standby 2024-04-25T00:27:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:27:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:53 smithi040 ceph-mon[32945]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:53 smithi040 ceph-mon[32945]: Deploying daemon mds.d on smithi031 2024-04-25T00:27:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:53 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] up:boot 2024-04-25T00:27:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:53 smithi040 ceph-mon[32945]: fsmap 3 up:standby 2024-04-25T00:27:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:27:53.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 -- 172.21.15.31:0/1100583729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81016b0 msgr2=0x7f2ee8101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 --2- 172.21.15.31:0/1100583729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81016b0 0x7f2ee8101b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009900 tx=0x7f2ed002a410 comp rx=0 tx=0).stop 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 -- 172.21.15.31:0/1100583729 shutdown_connections 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 --2- 172.21.15.31:0/1100583729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee8102070 0x7f2ee8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 --2- 172.21.15.31:0/1100583729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81016b0 0x7f2ee8101b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 --2- 172.21.15.31:0/1100583729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ee81004c0 0x7f2ee81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 -- 172.21.15.31:0/1100583729 >> 172.21.15.31:0/1100583729 conn(0x7f2ee80fba50 msgr2=0x7f2ee80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 -- 172.21.15.31:0/1100583729 shutdown_connections 2024-04-25T00:27:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.834+0000 7f2ee759e640 1 -- 172.21.15.31:0/1100583729 wait complete. 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 Processor -- start 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 -- start start 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 0x7f2ee8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ee81016b0 0x7f2ee8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee8102070 0x7f2ee819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810c000 con 0x7f2ee81004c0 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810be80 con 0x7f2ee8102070 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810c180 con 0x7f2ee81016b0 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 0x7f2ee8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 0x7f2ee8197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43526/0 (socket says 172.21.15.31:43526) 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee659c640 1 -- 172.21.15.31:0/660021109 learned_addr learned my addr 172.21.15.31:0/660021109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2edffff640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ee81016b0 0x7f2ee8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.835+0000 7f2ee6d9d640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee8102070 0x7f2ee819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee659c640 1 -- 172.21.15.31:0/660021109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ee81016b0 msgr2=0x7f2ee8197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee659c640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ee81016b0 0x7f2ee8197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee659c640 1 -- 172.21.15.31:0/660021109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee8102070 msgr2=0x7f2ee819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee659c640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee8102070 0x7f2ee819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee659c640 1 -- 172.21.15.31:0/660021109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee819f380 con 0x7f2ee81004c0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee659c640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 0x7f2ee8197760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c970 tx=0x7f2ed800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8007c10 con 0x7f2ee81004c0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed8007db0 con 0x7f2ee81004c0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee819f670 con 0x7f2ee81004c0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800f870 con 0x7f2ee81004c0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.836+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee819fb90 con 0x7f2ee81004c0 2024-04-25T00:27:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.837+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee819fe50 con 0x7f2ee81004c0 2024-04-25T00:27:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.838+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2ed800fa10 con 0x7f2ee81004c0 2024-04-25T00:27:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.839+0000 7f2edf7fe640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2ed40a2b90 0x7f2ed40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.839+0000 7f2edffff640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2ed40a2b90 0x7f2ed40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.839+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f2ed80cef30 con 0x7f2ee81004c0 2024-04-25T00:27:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.839+0000 7f2edffff640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2ed40a2b90 0x7f2ed40a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006010 tx=0x7f2ed0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:53.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:53.843+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ed809c320 con 0x7f2ee81004c0 2024-04-25T00:27:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.078+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ee81a0680 con 0x7f2ee81004c0 2024-04-25T00:27:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.079+0000 7f2edf7fe640 1 -- 172.21.15.31:0/660021109 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f2ed809bc20 con 0x7f2ee81004c0 2024-04-25T00:27:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:54.080 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T00:27:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2ed40a2b90 msgr2=0x7f2ed40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2ed40a2b90 0x7f2ed40a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006010 tx=0x7f2ed0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 msgr2=0x7f2ee8197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 0x7f2ee8197760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c970 tx=0x7f2ed800ce30 comp rx=0 tx=0).stop 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 shutdown_connections 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee8102070 0x7f2ee819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ee81016b0 0x7f2ee8197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2ed40a2b90 0x7f2ed40a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 --2- 172.21.15.31:0/660021109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee81004c0 0x7f2ee8197760 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 >> 172.21.15.31:0/660021109 conn(0x7f2ee80fba50 msgr2=0x7f2ee80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 shutdown_connections 2024-04-25T00:27:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:54.082+0000 7f2ee759e640 1 -- 172.21.15.31:0/660021109 wait complete. 2024-04-25T00:27:54.511 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 systemd[1]: Starting Ceph mds.d for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:27:54.772 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 podman[67247]: 2024-04-25 00:27:54.413578587 +0000 UTC m=+0.018574287 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:27:54.772 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 podman[67247]: 2024-04-25 00:27:54.542713556 +0000 UTC m=+0.147709281 container create 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T00:27:55.035 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 podman[67247]: 2024-04-25 00:27:54.778521023 +0000 UTC m=+0.383516712 container init 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:27:55.035 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 podman[67247]: 2024-04-25 00:27:54.781207101 +0000 UTC m=+0.386202783 container start 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:27:55.036 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 ceph-mds[67266]: -- 172.21.15.31:0/187857845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbe31f3a40 con 0x55dbe31c6800 2024-04-25T00:27:55.036 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d[67262]: starting mds.d at 2024-04-25T00:27:55.036 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 bash[67247]: 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 2024-04-25T00:27:55.036 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:27:54 smithi031 systemd[1]: Started Ceph mds.d for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:27:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/660021109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:55.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:55.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:55 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/660021109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:55.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:55.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:55 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/660021109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:55.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:55.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:55.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:55 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:56.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 -- 172.21.15.31:0/4236746237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 msgr2=0x7f357c106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 --2- 172.21.15.31:0/4236746237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c106fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3564009900 tx=0x7f356402a410 comp rx=0 tx=0).stop 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 -- 172.21.15.31:0/4236746237 shutdown_connections 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 --2- 172.21.15.31:0/4236746237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c107520 0x7f357c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 --2- 172.21.15.31:0/4236746237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c106fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 --2- 172.21.15.31:0/4236746237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c1022c0 0x7f357c1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 -- 172.21.15.31:0/4236746237 >> 172.21.15.31:0/4236746237 conn(0x7f357c0fbc30 msgr2=0x7f357c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:56.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 -- 172.21.15.31:0/4236746237 shutdown_connections 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 -- 172.21.15.31:0/4236746237 wait complete. 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.215+0000 7f3581a41640 1 Processor -- start 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 -- start start 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c1022c0 0x7f357c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c107520 0x7f357c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f357c10d320 con 0x7f357c104bf0 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f357c10d1a0 con 0x7f357c107520 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3581a41640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f357c10d4a0 con 0x7f357c1022c0 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43580/0 (socket says 172.21.15.31:43580) 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 -- 172.21.15.31:0/954681725 learned_addr learned my addr 172.21.15.31:0/954681725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f357b7fe640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c107520 0x7f357c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f357affd640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c1022c0 0x7f357c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 -- 172.21.15.31:0/954681725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c1022c0 msgr2=0x7f357c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c1022c0 0x7f357c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 -- 172.21.15.31:0/954681725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c107520 msgr2=0x7f357c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c107520 0x7f357c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 -- 172.21.15.31:0/954681725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c19f5f0 con 0x7f357c104bf0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3573fff640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c197f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35640314e0 tx=0x7f3564003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.216+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356403f070 con 0x7f357c104bf0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.217+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3564031c40 con 0x7f357c104bf0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.217+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357c19f880 con 0x7f357c104bf0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.217+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564035030 con 0x7f357c104bf0 2024-04-25T00:27:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.217+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357c19fd70 con 0x7f357c104bf0 2024-04-25T00:27:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.218+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357c0fd720 con 0x7f357c104bf0 2024-04-25T00:27:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.219+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3564042070 con 0x7f357c104bf0 2024-04-25T00:27:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.219+0000 7f3580a3f640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35680a2b90 0x7f35680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.220+0000 7f357affd640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35680a2b90 0x7f35680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.220+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f35640ef3d0 con 0x7f357c104bf0 2024-04-25T00:27:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.220+0000 7f357affd640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35680a2b90 0x7f35680a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f356c009bd0 tx=0x7f356c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.224+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35640bc0e0 con 0x7f357c104bf0 2024-04-25T00:27:56.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:56 smithi169 ceph-mon[33112]: Deploying daemon mds.e on smithi040 2024-04-25T00:27:56.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:56 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6836/951372407,v1:172.21.15.31:6837/951372407] up:boot 2024-04-25T00:27:56.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:56 smithi169 ceph-mon[33112]: fsmap 4 up:standby 2024-04-25T00:27:56.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:27:56.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:56 smithi040 ceph-mon[32945]: Deploying daemon mds.e on smithi040 2024-04-25T00:27:56.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:56 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6836/951372407,v1:172.21.15.31:6837/951372407] up:boot 2024-04-25T00:27:56.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:56 smithi040 ceph-mon[32945]: fsmap 4 up:standby 2024-04-25T00:27:56.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:27:56.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.450+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f357c062340 con 0x7f357c104bf0 2024-04-25T00:27:56.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.451+0000 7f3580a3f640 1 -- 172.21.15.31:0/954681725 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f35640c03e0 con 0x7f357c104bf0 2024-04-25T00:27:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:56.451 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T00:27:56.452 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2024-04-25T00:27:56.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35680a2b90 msgr2=0x7f35680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:56.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35680a2b90 0x7f35680a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f356c009bd0 tx=0x7f356c005f90 comp rx=0 tx=0).stop 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 msgr2=0x7f357c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c197f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35640314e0 tx=0x7f3564003c10 comp rx=0 tx=0).stop 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 shutdown_connections 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c107520 0x7f357c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35680a2b90 0x7f35680a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f357c104bf0 0x7f357c197f10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 --2- 172.21.15.31:0/954681725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c1022c0 0x7f357c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 >> 172.21.15.31:0/954681725 conn(0x7f357c0fbc30 msgr2=0x7f357c102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 shutdown_connections 2024-04-25T00:27:56.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:56.453+0000 7f3581a41640 1 -- 172.21.15.31:0/954681725 wait complete. 2024-04-25T00:27:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:56 smithi031 ceph-mon[29167]: Deploying daemon mds.e on smithi040 2024-04-25T00:27:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:56 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6836/951372407,v1:172.21.15.31:6837/951372407] up:boot 2024-04-25T00:27:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:56 smithi031 ceph-mon[29167]: fsmap 4 up:standby 2024-04-25T00:27:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:27:57.126 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:57 smithi040 ceph-mon[32945]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:57.127 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:57 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/954681725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:57 smithi169 ceph-mon[33112]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:57 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/954681725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:57.395 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 systemd[1]: Starting Ceph mds.e for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:27:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:57 smithi031 ceph-mon[29167]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:57 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/954681725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:57.837 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 podman[62501]: 2024-04-25 00:27:57.560163708 +0000 UTC m=+0.109311688 container create d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:27:57.838 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 podman[62501]: 2024-04-25 00:27:57.469694407 +0000 UTC m=+0.018842401 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:27:57.838 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 podman[62501]: 2024-04-25 00:27:57.838073703 +0000 UTC m=+0.387221683 container init d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-25T00:27:58.145 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 podman[62501]: 2024-04-25 00:27:57.841003421 +0000 UTC m=+0.390151402 container start d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:27:58.145 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 ceph-mds[62520]: -- 172.21.15.40:0/2787052526 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e9eb67a40 con 0x555e9eb3ac00 2024-04-25T00:27:58.145 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e[62516]: starting mds.e at 2024-04-25T00:27:58.145 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 bash[62501]: d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad 2024-04-25T00:27:58.145 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:27:57 smithi040 systemd[1]: Started Ceph mds.e for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:27:58.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 -- 172.21.15.31:0/1079266757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 msgr2=0x7fa984104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 --2- 172.21.15.31:0/1079266757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984104670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa974009900 tx=0x7fa97402a410 comp rx=0 tx=0).stop 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 -- 172.21.15.31:0/1079266757 shutdown_connections 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 --2- 172.21.15.31:0/1079266757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9841074e0 0x7fa9841098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 --2- 172.21.15.31:0/1079266757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa984104bb0 0x7fa984106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 --2- 172.21.15.31:0/1079266757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984104670 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 -- 172.21.15.31:0/1079266757 >> 172.21.15.31:0/1079266757 conn(0x7fa9840fbc30 msgr2=0x7fa9840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.581+0000 7fa988838640 1 -- 172.21.15.31:0/1079266757 shutdown_connections 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.582+0000 7fa988838640 1 -- 172.21.15.31:0/1079266757 wait complete. 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.582+0000 7fa988838640 1 Processor -- start 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.582+0000 7fa988838640 1 -- start start 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.582+0000 7fa988838640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.582+0000 7fa988838640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984104bb0 0x7fa984197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa988838640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9841074e0 0x7fa98419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa988838640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa98410d340 con 0x7fa984102280 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa988838640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa98410d1c0 con 0x7fa9841074e0 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa988838640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa98410d4c0 con 0x7fa984104bb0 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43588/0 (socket says 172.21.15.31:43588) 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 -- 172.21.15.31:0/265733285 learned_addr learned my addr 172.21.15.31:0/265733285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa97bfff640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984104bb0 0x7fa984197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa982d9d640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9841074e0 0x7fa98419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 -- 172.21.15.31:0/265733285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984104bb0 msgr2=0x7fa984197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984104bb0 0x7fa984197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 -- 172.21.15.31:0/265733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9841074e0 msgr2=0x7fa98419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9841074e0 0x7fa98419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 -- 172.21.15.31:0/265733285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98419f660 con 0x7fa984102280 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa982d9d640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9841074e0 0x7fa98419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa98259c640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984197a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740314e0 tx=0x7fa974003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97403f070 con 0x7fa984102280 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98419f8f0 con 0x7fa984102280 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa974031c40 con 0x7fa984102280 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974035030 con 0x7fa984102280 2024-04-25T00:27:58.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.583+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98419fde0 con 0x7fa984102280 2024-04-25T00:27:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.584+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9840fd720 con 0x7fa984102280 2024-04-25T00:27:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.586+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa974042070 con 0x7fa984102280 2024-04-25T00:27:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.586+0000 7fa97b7fe640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9700a2b90 0x7fa9700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.586+0000 7fa97bfff640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9700a2b90 0x7fa9700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.586+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa9740efad0 con 0x7fa984102280 2024-04-25T00:27:58.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.587+0000 7fa97bfff640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9700a2b90 0x7fa9700a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa984198f50 tx=0x7fa96c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:58.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.590+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa9740bced0 con 0x7fa984102280 2024-04-25T00:27:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.813+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa98402d170 con 0x7fa984102280 2024-04-25T00:27:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.814+0000 7fa97b7fe640 1 -- 172.21.15.31:0/265733285 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7fa9740bc7d0 con 0x7fa984102280 2024-04-25T00:27:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:27:58.815 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T00:27:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-04-25T00:27:58.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9700a2b90 msgr2=0x7fa9700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:58.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9700a2b90 0x7fa9700a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa984198f50 tx=0x7fa96c005f90 comp rx=0 tx=0).stop 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 msgr2=0x7fa984197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984197a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740314e0 tx=0x7fa974003c10 comp rx=0 tx=0).stop 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 shutdown_connections 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9841074e0 0x7fa98419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984104bb0 0x7fa984197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa9700a2b90 0x7fa9700a5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 --2- 172.21.15.31:0/265733285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984102280 0x7fa984197a40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 >> 172.21.15.31:0/265733285 conn(0x7fa9840fbc30 msgr2=0x7fa984102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 shutdown_connections 2024-04-25T00:27:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:27:58.816+0000 7fa988838640 1 -- 172.21.15.31:0/265733285 wait complete. 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6834/653343360,v1:172.21.15.40:6835/653343360] up:boot 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: fsmap 5 up:standby 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:59.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:59.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:59.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: Deploying daemon mds.f on smithi169 2024-04-25T00:27:59.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:27:58 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/265733285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6834/653343360,v1:172.21.15.40:6835/653343360] up:boot 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: fsmap 5 up:standby 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:59.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:59.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:59.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: Deploying daemon mds.f on smithi169 2024-04-25T00:27:59.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:27:58 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/265733285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6834/653343360,v1:172.21.15.40:6835/653343360] up:boot 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: fsmap 5 up:standby 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:27:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:27:59.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:27:59.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:27:59.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:59.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: Deploying daemon mds.f on smithi169 2024-04-25T00:27:59.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:27:58 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/265733285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:00.390 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 systemd[1]: Starting Ceph mds.f for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:28:00.697 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 podman[63733]: 2024-04-25 00:28:00.340904635 +0000 UTC m=+0.018042337 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:00.697 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 podman[63733]: 2024-04-25 00:28:00.46010614 +0000 UTC m=+0.137243808 container create ecbb7508dd8a3091f1fff239b7d30a5afee546723358fd8eefced9de2fbe7069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T00:28:00.697 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 podman[63733]: 2024-04-25 00:28:00.671291219 +0000 UTC m=+0.348428896 container init ecbb7508dd8a3091f1fff239b7d30a5afee546723358fd8eefced9de2fbe7069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:28:00.697 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 podman[63733]: 2024-04-25 00:28:00.673918827 +0000 UTC m=+0.351056505 container start ecbb7508dd8a3091f1fff239b7d30a5afee546723358fd8eefced9de2fbe7069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T00:28:00.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 -- 172.21.15.31:0/3219970603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8109660 msgr2=0x7f48f810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 --2- 172.21.15.31:0/3219970603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8109660 0x7f48f810baf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ca0 tx=0x7f48e802a950 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 -- 172.21.15.31:0/3219970603 shutdown_connections 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 --2- 172.21.15.31:0/3219970603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8109660 0x7f48f810baf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 --2- 172.21.15.31:0/3219970603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f8074120 0x7f48f8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 --2- 172.21.15.31:0/3219970603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48f8075700 0x7f48f8073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 -- 172.21.15.31:0/3219970603 >> 172.21.15.31:0/3219970603 conn(0x7f48f80fcd40 msgr2=0x7f48f80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 -- 172.21.15.31:0/3219970603 shutdown_connections 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 -- 172.21.15.31:0/3219970603 wait complete. 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 Processor -- start 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.941+0000 7f48fd542640 1 -- start start 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 0x7f48f8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f8075700 0x7f48f819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48f8109660 0x7f48f81a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48f810f4d0 con 0x7f48f8074120 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48f810f350 con 0x7f48f8109660 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f48f810f650 con 0x7f48f8075700 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 0x7f48f8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 0x7f48f8199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43610/0 (socket says 172.21.15.31:43610) 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 -- 172.21.15.31:0/3541443564 learned_addr learned my addr 172.21.15.31:0/3541443564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f67fc640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f8075700 0x7f48f819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f77fe640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48f8109660 0x7f48f81a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 -- 172.21.15.31:0/3541443564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f8075700 msgr2=0x7f48f819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f8075700 0x7f48f819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 -- 172.21.15.31:0/3541443564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48f8109660 msgr2=0x7f48f81a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48f8109660 0x7f48f81a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 -- 172.21.15.31:0/3541443564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f81a17b0 con 0x7f48f8074120 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48f6ffd640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 0x7f48f8199b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c890 tx=0x7f48f000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0004220 con 0x7f48f8074120 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f81a1aa0 con 0x7f48f8074120 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48f00043c0 con 0x7f48f8074120 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0004a00 con 0x7f48f8074120 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.942+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f81a2040 con 0x7f48f8074120 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.944+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f8108420 con 0x7f48f8074120 2024-04-25T00:28:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.945+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48f0004ba0 con 0x7f48f8074120 2024-04-25T00:28:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.946+0000 7f48dffff640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f48e00a2b90 0x7f48e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.946+0000 7f48f67fc640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f48e00a2b90 0x7f48e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.946+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f48f00ce860 con 0x7f48f8074120 2024-04-25T00:28:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.946+0000 7f48f67fc640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f48e00a2b90 0x7f48e00a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48f819b0a0 tx=0x7f48ec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:00.950+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48f0014030 con 0x7f48f8074120 2024-04-25T00:28:01.001 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 ceph-mds[63752]: -- 172.21.15.169:0/4083315154 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f3f995a40 con 0x556f3f968c00 2024-04-25T00:28:01.002 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f[63748]: starting mds.f at 2024-04-25T00:28:01.002 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 bash[63733]: ecbb7508dd8a3091f1fff239b7d30a5afee546723358fd8eefced9de2fbe7069 2024-04-25T00:28:01.002 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:28:00 smithi169 systemd[1]: Started Ceph mds.f for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:01.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.174+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48f802d3d0 con 0x7f48f8074120 2024-04-25T00:28:01.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.174+0000 7f48dffff640 1 -- 172.21.15.31:0/3541443564 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f48f009bc60 con 0x7f48f8074120 2024-04-25T00:28:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:01.175 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T00:28:01.175 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f48e00a2b90 msgr2=0x7f48e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f48e00a2b90 0x7f48e00a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48f819b0a0 tx=0x7f48ec00b390 comp rx=0 tx=0).stop 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 msgr2=0x7f48f8199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 0x7f48f8199b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c890 tx=0x7f48f000cd50 comp rx=0 tx=0).stop 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 shutdown_connections 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48f8109660 0x7f48f81a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f8075700 0x7f48f819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f48e00a2b90 0x7f48e00a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 --2- 172.21.15.31:0/3541443564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f8074120 0x7f48f8199b90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 >> 172.21.15.31:0/3541443564 conn(0x7f48f80fcd40 msgr2=0x7f48f81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 shutdown_connections 2024-04-25T00:28:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:01.176+0000 7f48fd542640 1 -- 172.21.15.31:0/3541443564 wait complete. 2024-04-25T00:28:01.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:01.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:01.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:01.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:01 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:01 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:01 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:02.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:02 smithi031 ceph-mon[29167]: Deploying daemon mds.c on smithi169 2024-04-25T00:28:02.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:02 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6834/3258138877,v1:172.21.15.169:6835/3258138877] up:boot 2024-04-25T00:28:02.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:02 smithi031 ceph-mon[29167]: fsmap 6 up:standby 2024-04-25T00:28:02.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:02 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:28:02.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:02 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3541443564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:02.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:02 smithi169 ceph-mon[33112]: Deploying daemon mds.c on smithi169 2024-04-25T00:28:02.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:02 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6834/3258138877,v1:172.21.15.169:6835/3258138877] up:boot 2024-04-25T00:28:02.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:02 smithi169 ceph-mon[33112]: fsmap 6 up:standby 2024-04-25T00:28:02.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:02 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:28:02.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:02 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3541443564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:02 smithi040 ceph-mon[32945]: Deploying daemon mds.c on smithi169 2024-04-25T00:28:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:02 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6834/3258138877,v1:172.21.15.169:6835/3258138877] up:boot 2024-04-25T00:28:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:02 smithi040 ceph-mon[32945]: fsmap 6 up:standby 2024-04-25T00:28:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:02 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:28:02.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:02 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3541443564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:03 smithi169 ceph-mon[33112]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:03.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:03 smithi031 ceph-mon[29167]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 -- 172.21.15.31:0/1089659399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74102230 msgr2=0x7f0a74108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 --2- 172.21.15.31:0/1089659399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74102230 0x7f0a74108ac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009ca0 tx=0x7f0a6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 -- 172.21.15.31:0/1089659399 shutdown_connections 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 --2- 172.21.15.31:0/1089659399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74102230 0x7f0a74108ac0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 --2- 172.21.15.31:0/1089659399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a74101870 0x7f0a74101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 --2- 172.21.15.31:0/1089659399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a74100680 0x7f0a74100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 -- 172.21.15.31:0/1089659399 >> 172.21.15.31:0/1089659399 conn(0x7f0a740fbbf0 msgr2=0x7f0a740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 -- 172.21.15.31:0/1089659399 shutdown_connections 2024-04-25T00:28:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.301+0000 7f0a7894f640 1 -- 172.21.15.31:0/1089659399 wait complete. 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 Processor -- start 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 -- start start 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 0x7f0a74197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a74101870 0x7f0a74197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a74102230 0x7f0a7419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410c1b0 con 0x7f0a74100680 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410c030 con 0x7f0a74101870 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.302+0000 7f0a7894f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410c330 con 0x7f0a74102230 2024-04-25T00:28:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 0x7f0a74197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 0x7f0a74197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54370/0 (socket says 172.21.15.31:54370) 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 -- 172.21.15.31:0/2036233078 learned_addr learned my addr 172.21.15.31:0/2036233078 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a6bfff640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a74101870 0x7f0a74197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a72d9d640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a74102230 0x7f0a7419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 -- 172.21.15.31:0/2036233078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a74102230 msgr2=0x7f0a7419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a74102230 0x7f0a7419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 -- 172.21.15.31:0/2036233078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a74101870 msgr2=0x7f0a74197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a74101870 0x7f0a74197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 -- 172.21.15.31:0/2036233078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7419f5b0 con 0x7f0a74100680 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7259c640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 0x7f0a74197990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400d950 tx=0x7f0a6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64017070 con 0x7f0a74100680 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a64009c30 con 0x7f0a74100680 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7419f8a0 con 0x7f0a74100680 2024-04-25T00:28:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6400f620 con 0x7f0a74100680 2024-04-25T00:28:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.303+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7419fda0 con 0x7f0a74100680 2024-04-25T00:28:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.304+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7419ffa0 con 0x7f0a74100680 2024-04-25T00:28:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.306+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0a6401a070 con 0x7f0a74100680 2024-04-25T00:28:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.306+0000 7f0a6b7fe640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0a600a2b90 0x7f0a600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.306+0000 7f0a6bfff640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0a600a2b90 0x7f0a600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:03.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.306+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0a640ce630 con 0x7f0a74100680 2024-04-25T00:28:03.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.307+0000 7f0a6bfff640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0a600a2b90 0x7f0a600a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a74198ea0 tx=0x7f0a5c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.310+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a64009de0 con 0x7f0a74100680 2024-04-25T00:28:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:03 smithi040 ceph-mon[32945]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.534+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a741a0170 con 0x7f0a74100680 2024-04-25T00:28:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.535+0000 7f0a6b7fe640 1 -- 172.21.15.31:0/2036233078 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f0a6409ba30 con 0x7f0a74100680 2024-04-25T00:28:03.536 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:03.536 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T00:28:03.536 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-04-25T00:28:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0a600a2b90 msgr2=0x7f0a600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0a600a2b90 0x7f0a600a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a74198ea0 tx=0x7f0a5c00b390 comp rx=0 tx=0).stop 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 msgr2=0x7f0a74197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 0x7f0a74197990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400d950 tx=0x7f0a6400de10 comp rx=0 tx=0).stop 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 shutdown_connections 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a74102230 0x7f0a7419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a74101870 0x7f0a74197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0a600a2b90 0x7f0a600a5050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 --2- 172.21.15.31:0/2036233078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a74100680 0x7f0a74197990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 >> 172.21.15.31:0/2036233078 conn(0x7f0a740fbbf0 msgr2=0x7f0a740fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 shutdown_connections 2024-04-25T00:28:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:03.537+0000 7f0a7894f640 1 -- 172.21.15.31:0/2036233078 wait complete. 2024-04-25T00:28:03.890 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:28:03 smithi169 systemd[1]: Started Ceph mds.c for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:04.380 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2036233078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:04.380 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.380 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.380 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.380 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:04.381 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:04.381 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:04 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:04.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2036233078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:04.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:04.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:04.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:04 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2036233078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:04.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:04 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:05.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:05 smithi169 ceph-mon[33112]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:05.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:05 smithi169 ceph-mon[33112]: Deploying daemon mds.h on smithi040 2024-04-25T00:28:05.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:05 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] up:boot 2024-04-25T00:28:05.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:05 smithi169 ceph-mon[33112]: fsmap 7 up:standby 2024-04-25T00:28:05.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:05 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:28:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:05 smithi040 ceph-mon[32945]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:05 smithi040 ceph-mon[32945]: Deploying daemon mds.h on smithi040 2024-04-25T00:28:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:05 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] up:boot 2024-04-25T00:28:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:05 smithi040 ceph-mon[32945]: fsmap 7 up:standby 2024-04-25T00:28:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:05 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:28:05.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:05 smithi031 ceph-mon[29167]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:05.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:05 smithi031 ceph-mon[29167]: Deploying daemon mds.h on smithi040 2024-04-25T00:28:05.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:05 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] up:boot 2024-04-25T00:28:05.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:05 smithi031 ceph-mon[29167]: fsmap 7 up:standby 2024-04-25T00:28:05.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:05 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:28:05.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.653+0000 7fc7088a7640 1 -- 172.21.15.31:0/3823937560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 msgr2=0x7fc704100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.653+0000 7fc7088a7640 1 --2- 172.21.15.31:0/3823937560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704100aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009900 tx=0x7fc6f402a410 comp rx=0 tx=0).stop 2024-04-25T00:28:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 -- 172.21.15.31:0/3823937560 shutdown_connections 2024-04-25T00:28:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 --2- 172.21.15.31:0/3823937560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc704102230 0x7fc704108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 --2- 172.21.15.31:0/3823937560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc704101870 0x7fc704101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 --2- 172.21.15.31:0/3823937560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704100aa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 -- 172.21.15.31:0/3823937560 >> 172.21.15.31:0/3823937560 conn(0x7fc7040fbbf0 msgr2=0x7fc7040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 -- 172.21.15.31:0/3823937560 shutdown_connections 2024-04-25T00:28:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 -- 172.21.15.31:0/3823937560 wait complete. 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 Processor -- start 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 -- start start 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc704101870 0x7fc704197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.654+0000 7fc7088a7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc704102230 0x7fc70419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc7088a7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410c180 con 0x7fc704100680 2024-04-25T00:28:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc7088a7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410c000 con 0x7fc704101870 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc7088a7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410c300 con 0x7fc704102230 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54396/0 (socket says 172.21.15.31:54396) 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 -- 172.21.15.31:0/118257454 learned_addr learned my addr 172.21.15.31:0/118257454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc702d9d640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc704102230 0x7fc70419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc701d9b640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc704101870 0x7fc704197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 -- 172.21.15.31:0/118257454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc704102230 msgr2=0x7fc70419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc704102230 0x7fc70419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 -- 172.21.15.31:0/118257454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc704101870 msgr2=0x7fc704197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc704101870 0x7fc704197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 -- 172.21.15.31:0/118257454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70419f6c0 con 0x7fc704100680 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc70259c640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704197aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40314e0 tx=0x7fc6f4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f403f030 con 0x7fc704100680 2024-04-25T00:28:05.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6f4003aa0 con 0x7fc704100680 2024-04-25T00:28:05.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70419f950 con 0x7fc704100680 2024-04-25T00:28:05.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4035070 con 0x7fc704100680 2024-04-25T00:28:05.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.655+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70419fe40 con 0x7fc704100680 2024-04-25T00:28:05.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.656+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7041a0010 con 0x7fc704100680 2024-04-25T00:28:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.658+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc6f4042030 con 0x7fc704100680 2024-04-25T00:28:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.659+0000 7fc6fb7fe640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6f00a2b90 0x7fc6f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.659+0000 7fc701d9b640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6f00a2b90 0x7fc6f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.659+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fc6f40ef570 con 0x7fc704100680 2024-04-25T00:28:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.659+0000 7fc701d9b640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6f00a2b90 0x7fc6f00a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc704198fb0 tx=0x7fc6ec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.663+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6f40bc280 con 0x7fc704100680 2024-04-25T00:28:05.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.889+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7041a03a0 con 0x7fc704100680 2024-04-25T00:28:05.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.890+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/118257454 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7fc6f40c0580 con 0x7fc704100680 2024-04-25T00:28:05.891 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:05.891 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:28:05.891 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6f00a2b90 msgr2=0x7fc6f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6f00a2b90 0x7fc6f00a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc704198fb0 tx=0x7fc6ec00b390 comp rx=0 tx=0).stop 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 msgr2=0x7fc704197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704197aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40314e0 tx=0x7fc6f4031a30 comp rx=0 tx=0).stop 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 shutdown_connections 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc704102230 0x7fc70419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc704101870 0x7fc704197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6f00a2b90 0x7fc6f00a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 --2- 172.21.15.31:0/118257454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc704100680 0x7fc704197aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 >> 172.21.15.31:0/118257454 conn(0x7fc7040fbbf0 msgr2=0x7fc704068960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 shutdown_connections 2024-04-25T00:28:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:05.892+0000 7fc7088a7640 1 -- 172.21.15.31:0/118257454 wait complete. 2024-04-25T00:28:06.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:06 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/118257454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:06.145 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 systemd[1]: Starting Ceph mds.h for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:28:06.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:06 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/118257454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:06.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:06 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/118257454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:06.537 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 podman[62812]: 2024-04-25 00:28:06.275035457 +0000 UTC m=+0.102682578 container create 1eaedcfead6aaf39d79b5f947eabdd94ce4489bc8a22a692dc006dd80ad7b243 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:28:06.537 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 podman[62812]: 2024-04-25 00:28:06.190862945 +0000 UTC m=+0.018510092 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:06.537 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 podman[62812]: 2024-04-25 00:28:06.511253489 +0000 UTC m=+0.338900610 container init 1eaedcfead6aaf39d79b5f947eabdd94ce4489bc8a22a692dc006dd80ad7b243 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T00:28:06.538 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 podman[62812]: 2024-04-25 00:28:06.513882617 +0000 UTC m=+0.341529746 container start 1eaedcfead6aaf39d79b5f947eabdd94ce4489bc8a22a692dc006dd80ad7b243 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T00:28:06.538 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 ceph-mds[62831]: -- 172.21.15.40:0/1209611692 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55967425fa40 con 0x559674232800 2024-04-25T00:28:06.895 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h[62827]: starting mds.h at 2024-04-25T00:28:06.895 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 bash[62812]: 1eaedcfead6aaf39d79b5f947eabdd94ce4489bc8a22a692dc006dd80ad7b243 2024-04-25T00:28:06.895 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:28:06 smithi040 systemd[1]: Started Ceph mds.h for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:07.338 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:07 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:07 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:07.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:07 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:07.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 -- 172.21.15.31:0/844709855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c101880 msgr2=0x7f1c7c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 --2- 172.21.15.31:0/844709855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c101880 0x7f1c7c101d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009900 tx=0x7f1c6402a410 comp rx=0 tx=0).stop 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 -- 172.21.15.31:0/844709855 shutdown_connections 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 --2- 172.21.15.31:0/844709855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7c102240 0x7f1c7c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 --2- 172.21.15.31:0/844709855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c101880 0x7f1c7c101d00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 --2- 172.21.15.31:0/844709855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c100690 0x7f1c7c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 -- 172.21.15.31:0/844709855 >> 172.21.15.31:0/844709855 conn(0x7f1c7c0fbc20 msgr2=0x7f1c7c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 -- 172.21.15.31:0/844709855 shutdown_connections 2024-04-25T00:28:08.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.020+0000 7f1c8125d640 1 -- 172.21.15.31:0/844709855 wait complete. 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 Processor -- start 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 -- start start 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7c100690 0x7f1c7c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c101880 0x7f1c7c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 0x7f1c7c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10c1a0 con 0x7f1c7c102240 2024-04-25T00:28:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10c020 con 0x7f1c7c101880 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c8125d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10c320 con 0x7f1c7c100690 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c7b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 0x7f1c7c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c7b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 0x7f1c7c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54404/0 (socket says 172.21.15.31:54404) 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.021+0000 7f1c7b59e640 1 -- 172.21.15.31:0/778673649 learned_addr learned my addr 172.21.15.31:0/778673649 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7a59c640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c101880 0x7f1c7c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7ad9d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7c100690 0x7f1c7c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7b59e640 1 -- 172.21.15.31:0/778673649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7c100690 msgr2=0x7f1c7c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7b59e640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7c100690 0x7f1c7c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7b59e640 1 -- 172.21.15.31:0/778673649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c101880 msgr2=0x7f1c7c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7b59e640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c101880 0x7f1c7c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7b59e640 1 -- 172.21.15.31:0/778673649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c19f650 con 0x7f1c7c102240 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7a59c640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c101880 0x7f1c7c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c7b59e640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 0x7f1c7c19efe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fd0 tx=0x7f1c6c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c017070 con 0x7f1c7c102240 2024-04-25T00:28:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c19f940 con 0x7f1c7c102240 2024-04-25T00:28:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c003e20 con 0x7f1c7c102240 2024-04-25T00:28:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c010460 con 0x7f1c7c102240 2024-04-25T00:28:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.022+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c19fe90 con 0x7f1c7c102240 2024-04-25T00:28:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.023+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c1a0120 con 0x7f1c7c102240 2024-04-25T00:28:08.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.024+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1c6c003970 con 0x7f1c7c102240 2024-04-25T00:28:08.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.025+0000 7f1c73fff640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1c680a2b90 0x7f1c680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:08.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.025+0000 7f1c7ad9d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1c680a2b90 0x7f1c680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:08.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.025+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f1c6c0ce9f0 con 0x7f1c7c102240 2024-04-25T00:28:08.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.025+0000 7f1c7ad9d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1c680a2b90 0x7f1c680a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c1016e0 tx=0x7f1c7400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.030+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c6c09bdf0 con 0x7f1c7c102240 2024-04-25T00:28:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.256+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c7c1a02f0 con 0x7f1c7c102240 2024-04-25T00:28:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.256+0000 7f1c73fff640 1 -- 172.21.15.31:0/778673649 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f1c6c09b6f0 con 0x7f1c7c102240 2024-04-25T00:28:08.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:08.257 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:28:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-04-25T00:28:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1c680a2b90 msgr2=0x7f1c680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1c680a2b90 0x7f1c680a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c1016e0 tx=0x7f1c7400a2d0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 msgr2=0x7f1c7c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 0x7f1c7c19efe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fd0 tx=0x7f1c6c00b260 comp rx=0 tx=0).stop 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 shutdown_connections 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1c680a2b90 0x7f1c680a5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.258+0000 7f1c8125d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c7c102240 0x7f1c7c19efe0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.259+0000 7f1c8125d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c101880 0x7f1c7c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.259+0000 7f1c8125d640 1 --2- 172.21.15.31:0/778673649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7c100690 0x7f1c7c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.259+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 >> 172.21.15.31:0/778673649 conn(0x7f1c7c0fbc20 msgr2=0x7f1c7c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.259+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 shutdown_connections 2024-04-25T00:28:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:08.259+0000 7f1c8125d640 1 -- 172.21.15.31:0/778673649 wait complete. 2024-04-25T00:28:08.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:08 smithi031 ceph-mon[29167]: Deploying daemon mds.i on smithi169 2024-04-25T00:28:08.269 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:08 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6836/3765244618,v1:172.21.15.40:6837/3765244618] up:boot 2024-04-25T00:28:08.269 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:08 smithi031 ceph-mon[29167]: fsmap 8 up:standby 2024-04-25T00:28:08.269 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:08 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:28:08.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:08 smithi169 ceph-mon[33112]: Deploying daemon mds.i on smithi169 2024-04-25T00:28:08.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:08 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6836/3765244618,v1:172.21.15.40:6837/3765244618] up:boot 2024-04-25T00:28:08.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:08 smithi169 ceph-mon[33112]: fsmap 8 up:standby 2024-04-25T00:28:08.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:08 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:28:08.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:08 smithi040 ceph-mon[32945]: Deploying daemon mds.i on smithi169 2024-04-25T00:28:08.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:08 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6836/3765244618,v1:172.21.15.40:6837/3765244618] up:boot 2024-04-25T00:28:08.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:08 smithi040 ceph-mon[32945]: fsmap 8 up:standby 2024-04-25T00:28:08.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:08 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:28:08.889 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:08 smithi169 systemd[1]: Starting Ceph mds.i for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:28:09.332 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:09 smithi169 ceph-mon[33112]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:09.332 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:09 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/778673649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:09.333 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 podman[64360]: 2024-04-25 00:28:08.938633004 +0000 UTC m=+0.017755759 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:09.333 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 podman[64360]: 2024-04-25 00:28:09.055146985 +0000 UTC m=+0.134269734 container create 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:28:09.333 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 podman[64360]: 2024-04-25 00:28:09.332859406 +0000 UTC m=+0.411982151 container init 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T00:28:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:09 smithi040 ceph-mon[32945]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:09 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/778673649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:09.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:09 smithi031 ceph-mon[29167]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:09.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:09 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/778673649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:09.639 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 podman[64360]: 2024-04-25 00:28:09.335457183 +0000 UTC m=+0.414579938 container start 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T00:28:09.640 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 ceph-mds[64379]: -- 172.21.15.169:0/3425232403 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556576581a40 con 0x556576554800 2024-04-25T00:28:09.640 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: starting mds.i at 2024-04-25T00:28:09.640 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 bash[64360]: 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc 2024-04-25T00:28:09.640 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:09 smithi169 systemd[1]: Started Ceph mds.i for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:10.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:10.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 -- 172.21.15.31:0/2705345325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc638102250 msgr2=0x7fc638108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:10.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 --2- 172.21.15.31:0/2705345325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc638102250 0x7fc638108ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ca0 tx=0x7fc63002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 -- 172.21.15.31:0/2705345325 shutdown_connections 2024-04-25T00:28:10.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 --2- 172.21.15.31:0/2705345325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc638102250 0x7fc638108ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 --2- 172.21.15.31:0/2705345325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 0x7fc638101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 --2- 172.21.15.31:0/2705345325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6381006a0 0x7fc638100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 -- 172.21.15.31:0/2705345325 >> 172.21.15.31:0/2705345325 conn(0x7fc6380fbc30 msgr2=0x7fc6380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 -- 172.21.15.31:0/2705345325 shutdown_connections 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.375+0000 7fc63f09a640 1 -- 172.21.15.31:0/2705345325 wait complete. 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 Processor -- start 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 -- start start 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 0x7fc638197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 0x7fc638197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc638102250 0x7fc63819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc63810c1d0 con 0x7fc6381006a0 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc63810c050 con 0x7fc638101890 2024-04-25T00:28:10.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63f09a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc63810c350 con 0x7fc638102250 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 0x7fc638197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 0x7fc638197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54418/0 (socket says 172.21.15.31:54418) 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 -- 172.21.15.31:0/1409210184 learned_addr learned my addr 172.21.15.31:0/1409210184 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc62ffff640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 0x7fc638197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 -- 172.21.15.31:0/1409210184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc638102250 msgr2=0x7fc63819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc638102250 0x7fc63819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 -- 172.21.15.31:0/1409210184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 msgr2=0x7fc638197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 0x7fc638197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc63ce36640 1 -- 172.21.15.31:0/1409210184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63819f590 con 0x7fc6381006a0 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.376+0000 7fc62ffff640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 0x7fc638197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.377+0000 7fc63ce36640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 0x7fc638197970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc63400d950 tx=0x7fc63400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.377+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634017070 con 0x7fc6381006a0 2024-04-25T00:28:10.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.377+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc634009c30 con 0x7fc6381006a0 2024-04-25T00:28:10.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.377+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63819f880 con 0x7fc6381006a0 2024-04-25T00:28:10.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.377+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63400f620 con 0x7fc6381006a0 2024-04-25T00:28:10.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.377+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63819fd80 con 0x7fc6381006a0 2024-04-25T00:28:10.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.378+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63819ffe0 con 0x7fc6381006a0 2024-04-25T00:28:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.379+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc63401a070 con 0x7fc6381006a0 2024-04-25T00:28:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.380+0000 7fc62dffb640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6200a2b90 0x7fc6200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.380+0000 7fc62ffff640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6200a2b90 0x7fc6200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.380+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fc6340ce750 con 0x7fc6381006a0 2024-04-25T00:28:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.380+0000 7fc62ffff640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6200a2b90 0x7fc6200a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc638198e80 tx=0x7fc628005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.384+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc634009de0 con 0x7fc6381006a0 2024-04-25T00:28:10.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.612+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6381a01b0 con 0x7fc6381006a0 2024-04-25T00:28:10.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.613+0000 7fc62dffb640 1 -- 172.21.15.31:0/1409210184 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7fc63409bab0 con 0x7fc6381006a0 2024-04-25T00:28:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:10.615 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.614+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6200a2b90 msgr2=0x7fc6200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.614+0000 7fc63f09a640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6200a2b90 0x7fc6200a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc638198e80 tx=0x7fc628005d30 comp rx=0 tx=0).stop 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.614+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 msgr2=0x7fc638197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.614+0000 7fc63f09a640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 0x7fc638197970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc63400d950 tx=0x7fc63400de10 comp rx=0 tx=0).stop 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 shutdown_connections 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc638102250 0x7fc63819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638101890 0x7fc638197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc6200a2b90 0x7fc6200a5050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 --2- 172.21.15.31:0/1409210184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6381006a0 0x7fc638197970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 >> 172.21.15.31:0/1409210184 conn(0x7fc6380fbc30 msgr2=0x7fc6380fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 shutdown_connections 2024-04-25T00:28:10.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:10.615+0000 7fc63f09a640 1 -- 172.21.15.31:0/1409210184 wait complete. 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: Deploying daemon mds.g on smithi031 2024-04-25T00:28:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] up:boot 2024-04-25T00:28:10.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: fsmap 9 up:standby 2024-04-25T00:28:10.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:10 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: Deploying daemon mds.g on smithi031 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] up:boot 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: fsmap 9 up:standby 2024-04-25T00:28:10.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:10 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: Deploying daemon mds.g on smithi031 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] up:boot 2024-04-25T00:28:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: fsmap 9 up:standby 2024-04-25T00:28:10.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:10 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:28:11.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:11 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1409210184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:11.773 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:11 smithi031 systemd[1]: Starting Ceph mds.g for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:28:11.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:11 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1409210184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:11.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:11 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1409210184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:12.216 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:11 smithi031 podman[67858]: 2024-04-25 00:28:11.844435022 +0000 UTC m=+0.018868345 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:12.216 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:11 smithi031 podman[67858]: 2024-04-25 00:28:11.971978622 +0000 UTC m=+0.146411935 container create d24fd09ac27caefedb4bc5d74ce308b260cd1db76cde9a4dc3926090a9b46a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T00:28:12.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:12.498 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:12 smithi031 podman[67858]: 2024-04-25 00:28:12.216607896 +0000 UTC m=+0.391041210 container init d24fd09ac27caefedb4bc5d74ce308b260cd1db76cde9a4dc3926090a9b46a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:28:12.498 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:12 smithi031 podman[67858]: 2024-04-25 00:28:12.219620268 +0000 UTC m=+0.394053590 container start d24fd09ac27caefedb4bc5d74ce308b260cd1db76cde9a4dc3926090a9b46a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:12.499 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mds[67877]: -- 172.21.15.31:0/773282897 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4c7619a40 con 0x55a4c75ecc00 2024-04-25T00:28:12.499 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g[67873]: starting mds.g at 2024-04-25T00:28:12.499 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:12 smithi031 bash[67858]: d24fd09ac27caefedb4bc5d74ce308b260cd1db76cde9a4dc3926090a9b46a23 2024-04-25T00:28:12.499 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:28:12 smithi031 systemd[1]: Started Ceph mds.g for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:12.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:12.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 -- 172.21.15.31:0/2114738094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 msgr2=0x7fd6a8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:12.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 --2- 172.21.15.31:0/2114738094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8108ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009ca0 tx=0x7fd6a002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:12.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 -- 172.21.15.31:0/2114738094 shutdown_connections 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 --2- 172.21.15.31:0/2114738094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8108ac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 --2- 172.21.15.31:0/2114738094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a8101870 0x7fd6a8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 --2- 172.21.15.31:0/2114738094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a8100680 0x7fd6a8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 -- 172.21.15.31:0/2114738094 >> 172.21.15.31:0/2114738094 conn(0x7fd6a80fbbf0 msgr2=0x7fd6a80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 -- 172.21.15.31:0/2114738094 shutdown_connections 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.732+0000 7fd6b0109640 1 -- 172.21.15.31:0/2114738094 wait complete. 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 Processor -- start 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 -- start start 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a8100680 0x7fd6a8104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a8101870 0x7fd6a8102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a810c1b0 con 0x7fd6a8102230 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a810c030 con 0x7fd6a8100680 2024-04-25T00:28:12.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6b0109640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a810c330 con 0x7fd6a8101870 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41326/0 (socket says 172.21.15.31:41326) 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 -- 172.21.15.31:0/1257873970 learned_addr learned my addr 172.21.15.31:0/1257873970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ad6a4640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a8101870 0x7fd6a8102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6adea5640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a8100680 0x7fd6a8104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 -- 172.21.15.31:0/1257873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a8101870 msgr2=0x7fd6a8102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a8101870 0x7fd6a8102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 -- 172.21.15.31:0/1257873970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a8100680 msgr2=0x7fd6a8104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a8100680 0x7fd6a8104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 -- 172.21.15.31:0/1257873970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a8103a20 con 0x7fd6a8102230 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.733+0000 7fd6ae6a6640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8103190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0031970 tx=0x7fd6a0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.734+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004040 con 0x7fd6a8102230 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.734+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6a0004690 con 0x7fd6a8102230 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.734+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a81a1ab0 con 0x7fd6a8102230 2024-04-25T00:28:12.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.734+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0030c60 con 0x7fd6a8102230 2024-04-25T00:28:12.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.734+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a81a2000 con 0x7fd6a8102230 2024-04-25T00:28:12.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.735+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a81a21d0 con 0x7fd6a8102230 2024-04-25T00:28:12.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.737+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6a00041e0 con 0x7fd6a8102230 2024-04-25T00:28:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.737+0000 7fd69effd640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd6940a2b90 0x7fd6940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.737+0000 7fd6adea5640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd6940a2b90 0x7fd6940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.738+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd6a00f01e0 con 0x7fd6a8102230 2024-04-25T00:28:12.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.738+0000 7fd6adea5640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd6940a2b90 0x7fd6940a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6a81016d0 tx=0x7fd6a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:12.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.742+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6a00bd540 con 0x7fd6a8102230 2024-04-25T00:28:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:12 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:12 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:12 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.982+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a81a23a0 con 0x7fd6a8102230 2024-04-25T00:28:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.983+0000 7fd69effd640 1 -- 172.21.15.31:0/1257873970 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fd6a0039e10 con 0x7fd6a8102230 2024-04-25T00:28:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:13.006 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:28:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd6940a2b90 msgr2=0x7fd6940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd6940a2b90 0x7fd6940a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6a81016d0 tx=0x7fd6a4005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 msgr2=0x7fd6a8103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8103190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0031970 tx=0x7fd6a0003ba0 comp rx=0 tx=0).stop 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 shutdown_connections 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd6940a2b90 0x7fd6940a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6a8102230 0x7fd6a8103190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a8101870 0x7fd6a8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 --2- 172.21.15.31:0/1257873970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a8100680 0x7fd6a8104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 >> 172.21.15.31:0/1257873970 conn(0x7fd6a80fbbf0 msgr2=0x7fd6a80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 shutdown_connections 2024-04-25T00:28:13.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:12.985+0000 7fd6b0109640 1 -- 172.21.15.31:0/1257873970 wait complete. 2024-04-25T00:28:13.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:13 smithi031 ceph-mon[29167]: Deploying daemon mds.a on smithi031 2024-04-25T00:28:13.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:13 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6838/2151696509,v1:172.21.15.31:6839/2151696509] up:boot 2024-04-25T00:28:13.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:13 smithi031 ceph-mon[29167]: fsmap 10 up:standby 2024-04-25T00:28:13.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:28:13.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:13 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1257873970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:13.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:13 smithi169 ceph-mon[33112]: Deploying daemon mds.a on smithi031 2024-04-25T00:28:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:13 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6838/2151696509,v1:172.21.15.31:6839/2151696509] up:boot 2024-04-25T00:28:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:13 smithi169 ceph-mon[33112]: fsmap 10 up:standby 2024-04-25T00:28:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:28:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:13 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1257873970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:13 smithi040 ceph-mon[32945]: Deploying daemon mds.a on smithi031 2024-04-25T00:28:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:13 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6838/2151696509,v1:172.21.15.31:6839/2151696509] up:boot 2024-04-25T00:28:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:13 smithi040 ceph-mon[32945]: fsmap 10 up:standby 2024-04-25T00:28:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:28:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:13 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1257873970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:14 smithi031 ceph-mon[29167]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:14.772 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:14 smithi031 systemd[1]: Starting Ceph mds.a for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:28:14.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:14 smithi169 ceph-mon[33112]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:14.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:14 smithi040 ceph-mon[32945]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:15.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:15.067 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:14 smithi031 podman[68199]: 2024-04-25 00:28:14.71947198 +0000 UTC m=+0.018755511 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:15.067 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:14 smithi031 podman[68199]: 2024-04-25 00:28:14.827544437 +0000 UTC m=+0.126827946 container create a44ed774e70d7a2280a3c8e3b83d823878e64ef126abe289eddc02bc8fffe350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:28:15.067 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:15 smithi031 podman[68199]: 2024-04-25 00:28:15.0384784 +0000 UTC m=+0.337761910 container init a44ed774e70d7a2280a3c8e3b83d823878e64ef126abe289eddc02bc8fffe350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:15.068 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:15 smithi031 podman[68199]: 2024-04-25 00:28:15.042178677 +0000 UTC m=+0.341462195 container start a44ed774e70d7a2280a3c8e3b83d823878e64ef126abe289eddc02bc8fffe350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:28:15.068 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:15 smithi031 ceph-mds[68237]: -- 172.21.15.31:0/1167031627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc3e403a40 con 0x55cc3e3d6c00 2024-04-25T00:28:15.068 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:15 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a[68213]: starting mds.a at 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 -- 172.21.15.31:0/263373504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08107460 msgr2=0x7fbf081098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/263373504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08107460 0x7fbf081098b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009ca0 tx=0x7fbf0002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 -- 172.21.15.31:0/263373504 shutdown_connections 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/263373504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08107460 0x7fbf081098b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/263373504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf08104b30 0x7fbf08106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/263373504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf08102200 0x7fbf081045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 -- 172.21.15.31:0/263373504 >> 172.21.15.31:0/263373504 conn(0x7fbf080fbbf0 msgr2=0x7fbf080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 -- 172.21.15.31:0/263373504 shutdown_connections 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.117+0000 7fbf0feb3640 1 -- 172.21.15.31:0/263373504 wait complete. 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 Processor -- start 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 -- start start 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 0x7fbf08101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf08104b30 0x7fbf081001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf08107460 0x7fbf081006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0810d2d0 con 0x7fbf08102200 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0810d150 con 0x7fbf08104b30 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0feb3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0810d450 con 0x7fbf08107460 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 0x7fbf08101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 0x7fbf08101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41368/0 (socket says 172.21.15.31:41368) 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 -- 172.21.15.31:0/4159760609 learned_addr learned my addr 172.21.15.31:0/4159760609 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0d44e640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf08104b30 0x7fbf081001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0e450640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf08107460 0x7fbf081006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 -- 172.21.15.31:0/4159760609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf08107460 msgr2=0x7fbf081006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf08107460 0x7fbf081006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 -- 172.21.15.31:0/4159760609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf08104b30 msgr2=0x7fbf081001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf08104b30 0x7fbf081001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 -- 172.21.15.31:0/4159760609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf08100f80 con 0x7fbf08102200 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbf0dc4f640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 0x7fbf08101b70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400e9f0 tx=0x7fbf0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400cd80 con 0x7fbf08102200 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.118+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf040054d0 con 0x7fbf08102200 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.119+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf081a1aa0 con 0x7fbf08102200 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.119+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04010490 con 0x7fbf08102200 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.119+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf081a1ff0 con 0x7fbf08102200 2024-04-25T00:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.119+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf080fd6e0 con 0x7fbf08102200 2024-04-25T00:28:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.121+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf04005020 con 0x7fbf08102200 2024-04-25T00:28:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.121+0000 7fbefeffd640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbef40a2c10 0x7fbef40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.121+0000 7fbf0d44e640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbef40a2c10 0x7fbef40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.121+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fbf040cef80 con 0x7fbf08102200 2024-04-25T00:28:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.122+0000 7fbf0d44e640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbef40a2c10 0x7fbef40a50d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf081018e0 tx=0x7fbef800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.126+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbf0409c380 con 0x7fbf08102200 2024-04-25T00:28:15.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.369+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf0802d170 con 0x7fbf08102200 2024-04-25T00:28:15.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.370+0000 7fbefeffd640 1 -- 172.21.15.31:0/4159760609 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fbf0409bc80 con 0x7fbf08102200 2024-04-25T00:28:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:28:15.371 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbef40a2c10 msgr2=0x7fbef40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbef40a2c10 0x7fbef40a50d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf081018e0 tx=0x7fbef800b390 comp rx=0 tx=0).stop 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 msgr2=0x7fbf08101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 0x7fbf08101b70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400e9f0 tx=0x7fbf0400edf0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 shutdown_connections 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf08107460 0x7fbf081006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf08104b30 0x7fbf081001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbef40a2c10 0x7fbef40a50d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 --2- 172.21.15.31:0/4159760609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf08102200 0x7fbf08101b70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 >> 172.21.15.31:0/4159760609 conn(0x7fbf080fbbf0 msgr2=0x7fbf08102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 shutdown_connections 2024-04-25T00:28:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:15.372+0000 7fbf0feb3640 1 -- 172.21.15.31:0/4159760609 wait complete. 2024-04-25T00:28:15.382 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:15 smithi031 bash[68199]: a44ed774e70d7a2280a3c8e3b83d823878e64ef126abe289eddc02bc8fffe350 2024-04-25T00:28:15.382 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:28:15 smithi031 systemd[1]: Started Ceph mds.a for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:16.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: Deploying daemon mds.b on smithi040 2024-04-25T00:28:16.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:16.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4159760609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:16.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6840/2045809592,v1:172.21.15.31:6841/2045809592] up:boot 2024-04-25T00:28:16.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: fsmap 11 up:standby 2024-04-25T00:28:16.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:16 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: Deploying daemon mds.b on smithi040 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4159760609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6840/2045809592,v1:172.21.15.31:6841/2045809592] up:boot 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: fsmap 11 up:standby 2024-04-25T00:28:16.620 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:16 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:28:16.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: Deploying daemon mds.b on smithi040 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4159760609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6840/2045809592,v1:172.21.15.31:6841/2045809592] up:boot 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: fsmap 11 up:standby 2024-04-25T00:28:16.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:16 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:28:17.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:17.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 -- 172.21.15.31:0/987004797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 msgr2=0x7f6ac4109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/987004797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac4109bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009ca0 tx=0x7f6abc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 -- 172.21.15.31:0/987004797 shutdown_connections 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/987004797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac4109bf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/987004797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41029a0 0x7f6ac4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/987004797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41017b0 0x7f6ac4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 -- 172.21.15.31:0/987004797 >> 172.21.15.31:0/987004797 conn(0x7f6ac40fcd20 msgr2=0x7f6ac40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 -- 172.21.15.31:0/987004797 shutdown_connections 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 -- 172.21.15.31:0/987004797 wait complete. 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 Processor -- start 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.499+0000 7f6acaa6d640 1 -- start start 2024-04-25T00:28:17.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 0x7f6ac4199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41029a0 0x7f6ac419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac410d2e0 con 0x7f6ac4103360 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac410d160 con 0x7f6ac41017b0 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac410d460 con 0x7f6ac41029a0 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac8809640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 0x7f6ac4199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac41a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41390/0 (socket says 172.21.15.31:41390) 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 -- 172.21.15.31:0/2435284514 learned_addr learned my addr 172.21.15.31:0/2435284514 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac8809640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 0x7f6ac4199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33972/0 (socket says 172.21.15.31:33972) 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac3fff640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41029a0 0x7f6ac419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 -- 172.21.15.31:0/2435284514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41029a0 msgr2=0x7f6ac419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41029a0 0x7f6ac419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 -- 172.21.15.31:0/2435284514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 msgr2=0x7f6ac4199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 0x7f6ac4199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 -- 172.21.15.31:0/2435284514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac41a1810 con 0x7f6ac4103360 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac8809640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 0x7f6ac4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac3fff640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41029a0 0x7f6ac419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac900a640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac41a11a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6abc031970 tx=0x7f6abc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc004040 con 0x7f6ac4103360 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac41a1aa0 con 0x7f6ac4103360 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6abc004690 con 0x7f6ac4103360 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.500+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc030c60 con 0x7f6ac4103360 2024-04-25T00:28:17.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.501+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41a2040 con 0x7f6ac4103360 2024-04-25T00:28:17.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.502+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac41a2210 con 0x7f6ac4103360 2024-04-25T00:28:17.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.503+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6abc0041e0 con 0x7f6ac4103360 2024-04-25T00:28:17.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.503+0000 7f6ac1ffb640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ab40a2b90 0x7f6ab40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:17.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.504+0000 7f6ac8809640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ab40a2b90 0x7f6ab40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:17.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.504+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6abc0f00d0 con 0x7f6ac4103360 2024-04-25T00:28:17.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.504+0000 7f6ac8809640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ab40a2b90 0x7f6ab40a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005fd0 tx=0x7f6ab800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.508+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6abc0bd4d0 con 0x7f6ac4103360 2024-04-25T00:28:17.645 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:17 smithi040 systemd[1]: Starting Ceph mds.b for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:28:17.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.731+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ac41a2500 con 0x7f6ac4103360 2024-04-25T00:28:17.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.732+0000 7f6ac1ffb640 1 -- 172.21.15.31:0/2435284514 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f6abc0bcdd0 con 0x7f6ac4103360 2024-04-25T00:28:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:17.733 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:28:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-04-25T00:28:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ab40a2b90 msgr2=0x7f6ab40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ab40a2b90 0x7f6ab40a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005fd0 tx=0x7f6ab800b390 comp rx=0 tx=0).stop 2024-04-25T00:28:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 msgr2=0x7f6ac41a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac41a11a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6abc031970 tx=0x7f6abc003ba0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 shutdown_connections 2024-04-25T00:28:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ab40a2b90 0x7f6ab40a5050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac4103360 0x7f6ac41a11a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac41029a0 0x7f6ac419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 --2- 172.21.15.31:0/2435284514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac41017b0 0x7f6ac4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 >> 172.21.15.31:0/2435284514 conn(0x7f6ac40fcd20 msgr2=0x7f6ac4073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 shutdown_connections 2024-04-25T00:28:17.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:17.734+0000 7f6acaa6d640 1 -- 172.21.15.31:0/2435284514 wait complete. 2024-04-25T00:28:18.051 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:17 smithi040 podman[63123]: 2024-04-25 00:28:17.686799008 +0000 UTC m=+0.018155924 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:18.051 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:17 smithi040 podman[63123]: 2024-04-25 00:28:17.807014548 +0000 UTC m=+0.138371507 container create f93e9953f001bb80ad27bfdaf443d8b68abda89bd1598f4587e4fbf52049c67b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T00:28:18.051 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:18 smithi040 podman[63123]: 2024-04-25 00:28:18.051413604 +0000 UTC m=+0.382770518 container init f93e9953f001bb80ad27bfdaf443d8b68abda89bd1598f4587e4fbf52049c67b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:28:18.359 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:18 smithi040 podman[63123]: 2024-04-25 00:28:18.054311772 +0000 UTC m=+0.385668689 container start f93e9953f001bb80ad27bfdaf443d8b68abda89bd1598f4587e4fbf52049c67b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T00:28:18.359 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mds[63142]: -- 172.21.15.40:0/2490676515 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2abcbfa40 con 0x55d2abc92c00 2024-04-25T00:28:18.359 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b[63138]: starting mds.b at 2024-04-25T00:28:18.359 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:18 smithi040 bash[63123]: f93e9953f001bb80ad27bfdaf443d8b68abda89bd1598f4587e4fbf52049c67b 2024-04-25T00:28:18.359 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:28:18 smithi040 systemd[1]: Started Ceph mds.b for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:28:18.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2435284514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:18 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2435284514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:18 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2435284514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:18.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:18 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:19.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:19 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] up:boot 2024-04-25T00:28:19.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:19 smithi169 ceph-mon[33112]: fsmap 12 up:standby 2024-04-25T00:28:19.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:19 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:28:19.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:19 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:19.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:19 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] up:boot 2024-04-25T00:28:19.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:19 smithi040 ceph-mon[32945]: fsmap 12 up:standby 2024-04-25T00:28:19.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:19 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:28:19.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:19 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:19.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:19.770 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:19 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] up:boot 2024-04-25T00:28:19.770 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:19 smithi031 ceph-mon[29167]: fsmap 12 up:standby 2024-04-25T00:28:19.770 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:19 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:28:19.770 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:19 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 -- 172.21.15.31:0/2583239668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5508104bd0 msgr2=0x7f5508106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 --2- 172.21.15.31:0/2583239668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5508104bd0 0x7f5508106fc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009900 tx=0x7f54fc02a410 comp rx=0 tx=0).stop 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 -- 172.21.15.31:0/2583239668 shutdown_connections 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 --2- 172.21.15.31:0/2583239668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 0x7f55081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 --2- 172.21.15.31:0/2583239668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5508104bd0 0x7f5508106fc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 --2- 172.21.15.31:0/2583239668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55081022a0 0x7f5508104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 -- 172.21.15.31:0/2583239668 >> 172.21.15.31:0/2583239668 conn(0x7f55080fbbf0 msgr2=0x7f55080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 -- 172.21.15.31:0/2583239668 shutdown_connections 2024-04-25T00:28:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.871+0000 7f550ca12640 1 -- 172.21.15.31:0/2583239668 wait complete. 2024-04-25T00:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 Processor -- start 2024-04-25T00:28:19.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 -- start start 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 0x7f5508197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5508104bd0 0x7f5508197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 0x7f550819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f550810d280 con 0x7f55081022a0 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f550810d100 con 0x7f5508104bd0 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550ca12640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f550810d400 con 0x7f5508107500 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 0x7f5508197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 0x7f5508197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41408/0 (socket says 172.21.15.31:41408) 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f550659c640 1 -- 172.21.15.31:0/2817607713 learned_addr learned my addr 172.21.15.31:0/2817607713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f5505d9b640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5508104bd0 0x7f5508197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.872+0000 7f5506d9d640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 0x7f550819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550659c640 1 -- 172.21.15.31:0/2817607713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 msgr2=0x7f550819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550659c640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 0x7f550819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550659c640 1 -- 172.21.15.31:0/2817607713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5508104bd0 msgr2=0x7f5508197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550659c640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5508104bd0 0x7f5508197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550659c640 1 -- 172.21.15.31:0/2817607713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550819f5f0 con 0x7f55081022a0 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f5506d9d640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 0x7f550819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550659c640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 0x7f5508197a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e9f0 tx=0x7f54f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f800cd80 con 0x7f55081022a0 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f80108e0 con 0x7f55081022a0 2024-04-25T00:28:19.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550819f8e0 con 0x7f55081022a0 2024-04-25T00:28:19.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8018920 con 0x7f55081022a0 2024-04-25T00:28:19.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.873+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550819fe50 con 0x7f55081022a0 2024-04-25T00:28:19.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.874+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55080fd6e0 con 0x7f55081022a0 2024-04-25T00:28:19.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.875+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f54f8010a90 con 0x7f55081022a0 2024-04-25T00:28:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.876+0000 7f54f77fe640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f54f00a2c10 0x7f54f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.876+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f54f80cef70 con 0x7f55081022a0 2024-04-25T00:28:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.876+0000 7f5505d9b640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f54f00a2c10 0x7f54f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.877+0000 7f5505d9b640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f54f00a2c10 0x7f54f00a50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f54fc004770 tx=0x7f54fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:19.881+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f54f809c2d0 con 0x7f55081022a0 2024-04-25T00:28:20.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.134+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f550802d170 con 0x7f55081022a0 2024-04-25T00:28:20.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.135+0000 7f54f77fe640 1 -- 172.21.15.31:0/2817607713 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f54f809bbd0 con 0x7f55081022a0 2024-04-25T00:28:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:20.136 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:28:20.136 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f54f00a2c10 msgr2=0x7f54f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f54f00a2c10 0x7f54f00a50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f54fc004770 tx=0x7f54fc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 msgr2=0x7f5508197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 0x7f5508197a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e9f0 tx=0x7f54f800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 shutdown_connections 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5508107500 0x7f550819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5508104bd0 0x7f5508197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f54f00a2c10 0x7f54f00a50d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 --2- 172.21.15.31:0/2817607713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55081022a0 0x7f5508197a20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 >> 172.21.15.31:0/2817607713 conn(0x7f55080fbbf0 msgr2=0x7f5508102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 shutdown_connections 2024-04-25T00:28:20.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.137+0000 7f550ca12640 1 -- 172.21.15.31:0/2817607713 wait complete. 2024-04-25T00:28:20.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:28:20.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.293+0000 7f6ae095a640 1 -- 172.21.15.31:0/3341158901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc107460 msgr2=0x7f6adc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.293+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3341158901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc107460 0x7f6adc1098b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009ca0 tx=0x7f6ad402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 -- 172.21.15.31:0/3341158901 shutdown_connections 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3341158901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc107460 0x7f6adc1098b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3341158901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6adc104b30 0x7f6adc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3341158901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc102200 0x7f6adc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 -- 172.21.15.31:0/3341158901 >> 172.21.15.31:0/3341158901 conn(0x7f6adc0fbbf0 msgr2=0x7f6adc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 -- 172.21.15.31:0/3341158901 shutdown_connections 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 -- 172.21.15.31:0/3341158901 wait complete. 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 Processor -- start 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.294+0000 7f6ae095a640 1 -- start start 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6adc102200 0x7f6adc197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 0x7f6adc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107460 0x7f6adc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10d2d0 con 0x7f6adc104b30 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10d150 con 0x7f6adc102200 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10d450 con 0x7f6adc107460 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 0x7f6adc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 0x7f6adc197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41418/0 (socket says 172.21.15.31:41418) 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 -- 172.21.15.31:0/3400002599 learned_addr learned my addr 172.21.15.31:0/3400002599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ada59c640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6adc102200 0x7f6adc197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6adad9d640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107460 0x7f6adc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 -- 172.21.15.31:0/3400002599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107460 msgr2=0x7f6adc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107460 0x7f6adc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 -- 172.21.15.31:0/3400002599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6adc102200 msgr2=0x7f6adc197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6adc102200 0x7f6adc197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 -- 172.21.15.31:0/3400002599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc19f5a0 con 0x7f6adc104b30 2024-04-25T00:28:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ad9d9b640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 0x7f6adc197ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000e9c0 tx=0x7f6ad000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000cd80 con 0x7f6adc104b30 2024-04-25T00:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc19f890 con 0x7f6adc104b30 2024-04-25T00:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad00054d0 con 0x7f6adc104b30 2024-04-25T00:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0010430 con 0x7f6adc104b30 2024-04-25T00:28:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.295+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc19fde0 con 0x7f6adc104b30 2024-04-25T00:28:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.297+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc0fd6e0 con 0x7f6adc104b30 2024-04-25T00:28:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.298+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6ad0004fd0 con 0x7f6adc104b30 2024-04-25T00:28:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.298+0000 7f6acb7fe640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ac40a2b90 0x7f6ac40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.298+0000 7f6ada59c640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ac40a2b90 0x7f6ac40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.298+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6ad00cefe0 con 0x7f6adc104b30 2024-04-25T00:28:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.299+0000 7f6ada59c640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ac40a2b90 0x7f6ac40a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009bd0 tx=0x7f6acc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.303+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6ad009c3e0 con 0x7f6adc104b30 2024-04-25T00:28:20.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.496+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6adc02d350 con 0x7f6adc104b30 2024-04-25T00:28:20.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.497+0000 7f6acb7fe640 1 -- 172.21.15.31:0/3400002599 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18446 (secure 0 0 0) 0x7f6ad009bce0 con 0x7f6adc104b30 2024-04-25T00:28:20.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:20.498 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:27:34.530272+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:28:20.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ac40a2b90 msgr2=0x7f6ac40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ac40a2b90 0x7f6ac40a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009bd0 tx=0x7f6acc005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 msgr2=0x7f6adc197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 0x7f6adc197ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000e9c0 tx=0x7f6ad000ee80 comp rx=0 tx=0).stop 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 shutdown_connections 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107460 0x7f6adc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6ac40a2b90 0x7f6ac40a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc104b30 0x7f6adc197ec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 --2- 172.21.15.31:0/3400002599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6adc102200 0x7f6adc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 >> 172.21.15.31:0/3400002599 conn(0x7f6adc0fbbf0 msgr2=0x7f6adc102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 shutdown_connections 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.499+0000 7f6ae095a640 1 -- 172.21.15.31:0/3400002599 wait complete. 2024-04-25T00:28:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2817607713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:20.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:20 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:28:20.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:28:20.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 -- 172.21.15.31:0/1308715232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 msgr2=0x7f6f5c104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 --2- 172.21.15.31:0/1308715232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c104690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009900 tx=0x7f6f5402a410 comp rx=0 tx=0).stop 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 -- 172.21.15.31:0/1308715232 shutdown_connections 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 --2- 172.21.15.31:0/1308715232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f5c107500 0x7f6f5c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 --2- 172.21.15.31:0/1308715232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f5c104bd0 0x7f6f5c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 --2- 172.21.15.31:0/1308715232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c104690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 -- 172.21.15.31:0/1308715232 >> 172.21.15.31:0/1308715232 conn(0x7f6f5c0fbc30 msgr2=0x7f6f5c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 -- 172.21.15.31:0/1308715232 shutdown_connections 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.625+0000 7f6f6104d640 1 -- 172.21.15.31:0/1308715232 wait complete. 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 Processor -- start 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 -- start start 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f5c104bd0 0x7f6f5c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f5c107500 0x7f6f5c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10d320 con 0x7f6f5c1022a0 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10d1a0 con 0x7f6f5c104bd0 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f6104d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10d4a0 con 0x7f6f5c107500 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41434/0 (socket says 172.21.15.31:41434) 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 -- 172.21.15.31:0/2679351796 learned_addr learned my addr 172.21.15.31:0/2679351796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5a59c640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f5c104bd0 0x7f6f5c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5b59e640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f5c107500 0x7f6f5c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 -- 172.21.15.31:0/2679351796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f5c107500 msgr2=0x7f6f5c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f5c107500 0x7f6f5c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 -- 172.21.15.31:0/2679351796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f5c104bd0 msgr2=0x7f6f5c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f5c104bd0 0x7f6f5c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.626+0000 7f6f5ad9d640 1 -- 172.21.15.31:0/2679351796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c19f650 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.627+0000 7f6f5ad9d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c197a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f540314e0 tx=0x7f6f54003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.627+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5403f070 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.627+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c19f8e0 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.627+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f54031c40 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.627+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54035030 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.627+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c19fe20 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.628+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c0fd720 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.629+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6f54042070 con 0x7f6f5c1022a0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.629+0000 7f6f53fff640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6f480a2b90 0x7f6f480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.630+0000 7f6f5a59c640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6f480a2b90 0x7f6f480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.630+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6f540ef2e0 con 0x7f6f5c1022a0 2024-04-25T00:28:20.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.630+0000 7f6f5a59c640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6f480a2b90 0x7f6f480a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c198f40 tx=0x7f6f44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:20.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.634+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f540bc090 con 0x7f6f5c1022a0 2024-04-25T00:28:20.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:20.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2817607713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:20.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:20.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:20 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2817607713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:20.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:20.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:20 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.820+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f5c062340 con 0x7f6f5c1022a0 2024-04-25T00:28:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.820+0000 7f6f53fff640 1 -- 172.21.15.31:0/2679351796 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f6f540c0390 con 0x7f6f5c1022a0 2024-04-25T00:28:20.821 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:28:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6f480a2b90 msgr2=0x7f6f480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6f480a2b90 0x7f6f480a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c198f40 tx=0x7f6f44005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 msgr2=0x7f6f5c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c197a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f540314e0 tx=0x7f6f54003c10 comp rx=0 tx=0).stop 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 shutdown_connections 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f5c107500 0x7f6f5c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f5c104bd0 0x7f6f5c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6f480a2b90 0x7f6f480a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 --2- 172.21.15.31:0/2679351796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c1022a0 0x7f6f5c197a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 >> 172.21.15.31:0/2679351796 conn(0x7f6f5c0fbc30 msgr2=0x7f6f5c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 shutdown_connections 2024-04-25T00:28:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.823+0000 7f6f6104d640 1 -- 172.21.15.31:0/2679351796 wait complete. 2024-04-25T00:28:20.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 -- 172.21.15.31:0/2864033586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614104b30 msgr2=0x7fe614106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 --2- 172.21.15.31:0/2864033586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614104b30 0x7fe614106f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009900 tx=0x7fe5fc02a410 comp rx=0 tx=0).stop 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 -- 172.21.15.31:0/2864033586 shutdown_connections 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 --2- 172.21.15.31:0/2864033586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe614107460 0x7fe6141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 --2- 172.21.15.31:0/2864033586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614104b30 0x7fe614106f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 --2- 172.21.15.31:0/2864033586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe614102200 0x7fe6141045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 -- 172.21.15.31:0/2864033586 >> 172.21.15.31:0/2864033586 conn(0x7fe6140fbbf0 msgr2=0x7fe6140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.936+0000 7fe61a328640 1 -- 172.21.15.31:0/2864033586 shutdown_connections 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 -- 172.21.15.31:0/2864033586 wait complete. 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 Processor -- start 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 -- start start 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe614102200 0x7fe614101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe614104b30 0x7fe6141001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 0x7fe6141006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe61410d2a0 con 0x7fe614107460 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe61410d120 con 0x7fe614104b30 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe61a328640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe61410d420 con 0x7fe614102200 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.937+0000 7fe6188c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 0x7fe6141006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 0x7fe6141006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41456/0 (socket says 172.21.15.31:41456) 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 -- 172.21.15.31:0/2181233000 learned_addr learned my addr 172.21.15.31:0/2181233000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:20.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe613fff640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe614102200 0x7fe614101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6137fe640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe614104b30 0x7fe6141001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 -- 172.21.15.31:0/2181233000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe614102200 msgr2=0x7fe614101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe614102200 0x7fe614101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 -- 172.21.15.31:0/2181233000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe614104b30 msgr2=0x7fe6141001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe614104b30 0x7fe6141001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 -- 172.21.15.31:0/2181233000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe614100fb0 con 0x7fe614107460 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6188c5640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 0x7fe6141006f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00eaa0 tx=0x7fe60c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00cdb0 con 0x7fe614107460 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe60c005020 con 0x7fe614107460 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6141a1b60 con 0x7fe614107460 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0116d0 con 0x7fe614107460 2024-04-25T00:28:20.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.938+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6141a2030 con 0x7fe614107460 2024-04-25T00:28:20.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.939+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6140fd6e0 con 0x7fe614107460 2024-04-25T00:28:20.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.940+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe60c0051d0 con 0x7fe614107460 2024-04-25T00:28:20.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.941+0000 7fe6117fa640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe6000a2b90 0x7fe6000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:20.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.941+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe60c0cf080 con 0x7fe614107460 2024-04-25T00:28:20.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.941+0000 7fe613fff640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe6000a2b90 0x7fe6000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:20.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.942+0000 7fe613fff640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe6000a2b90 0x7fe6000a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe604009bd0 tx=0x7fe604005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:20.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:20.945+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe60c016030 con 0x7fe614107460 2024-04-25T00:28:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.143+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe61402d3c0 con 0x7fe614107460 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.143+0000 7fe6117fa640 1 -- 172.21.15.31:0/2181233000 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7fe60c09be30 con 0x7fe614107460 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:28:21.144 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe6000a2b90 msgr2=0x7fe6000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe6000a2b90 0x7fe6000a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe604009bd0 tx=0x7fe604005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 msgr2=0x7fe6141006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 0x7fe6141006f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00eaa0 tx=0x7fe60c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 shutdown_connections 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe6000a2b90 0x7fe6000a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe614107460 0x7fe6141006f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe614104b30 0x7fe6141001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 --2- 172.21.15.31:0/2181233000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe614102200 0x7fe614101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.145+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 >> 172.21.15.31:0/2181233000 conn(0x7fe6140fbbf0 msgr2=0x7fe614102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.146+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 shutdown_connections 2024-04-25T00:28:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.146+0000 7fe61a328640 1 -- 172.21.15.31:0/2181233000 wait complete. 2024-04-25T00:28:21.157 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:28:21.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:28:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 -- 172.21.15.31:0/2254995284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 msgr2=0x7f19d8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 --2- 172.21.15.31:0/2254995284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d8101bc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009900 tx=0x7f19c802a410 comp rx=0 tx=0).stop 2024-04-25T00:28:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 -- 172.21.15.31:0/2254995284 shutdown_connections 2024-04-25T00:28:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 --2- 172.21.15.31:0/2254995284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d8103350 0x7f19d8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 --2- 172.21.15.31:0/2254995284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d8102990 0x7f19d8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 --2- 172.21.15.31:0/2254995284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d8101bc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 -- 172.21.15.31:0/2254995284 >> 172.21.15.31:0/2254995284 conn(0x7f19d80fcd10 msgr2=0x7f19d80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 -- 172.21.15.31:0/2254995284 shutdown_connections 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.267+0000 7f19de8b2640 1 -- 172.21.15.31:0/2254995284 wait complete. 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 Processor -- start 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 -- start start 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d8102990 0x7f19d8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d8103350 0x7f19d819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810d2d0 con 0x7f19d81017a0 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810d150 con 0x7f19d8103350 2024-04-25T00:28:21.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19de8b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810d450 con 0x7f19d8102990 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d81979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41474/0 (socket says 172.21.15.31:41474) 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 -- 172.21.15.31:0/3348028446 learned_addr learned my addr 172.21.15.31:0/3348028446 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d77fe640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d8102990 0x7f19d8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19dce4f640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d8103350 0x7f19d819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 -- 172.21.15.31:0/3348028446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d8102990 msgr2=0x7f19d8197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d8102990 0x7f19d8197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 -- 172.21.15.31:0/3348028446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d8103350 msgr2=0x7f19d819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d8103350 0x7f19d819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.268+0000 7f19d7fff640 1 -- 172.21.15.31:0/3348028446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d819f5c0 con 0x7f19d81017a0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.269+0000 7f19d7fff640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d81979a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19c80314e0 tx=0x7f19c80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.269+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c803f030 con 0x7f19d81017a0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.269+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d819f850 con 0x7f19d81017a0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.269+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c80044d0 con 0x7f19d81017a0 2024-04-25T00:28:21.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.269+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8035070 con 0x7f19d81017a0 2024-04-25T00:28:21.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.269+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d819fd90 con 0x7f19d81017a0 2024-04-25T00:28:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.270+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d8108ae0 con 0x7f19d81017a0 2024-04-25T00:28:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.271+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f19c8042030 con 0x7f19d81017a0 2024-04-25T00:28:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.271+0000 7f19d57fa640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f19c00a2b90 0x7f19c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.272+0000 7f19d77fe640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f19c00a2b90 0x7f19c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.272+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f19c80ef450 con 0x7f19d81017a0 2024-04-25T00:28:21.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.272+0000 7f19d77fe640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f19c00a2b90 0x7f19c00a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19d8198eb0 tx=0x7f19cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:21.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.276+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f19c80bc850 con 0x7f19d81017a0 2024-04-25T00:28:21.412 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3400002599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:21.412 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:21 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:21.412 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2679351796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:28:21.412 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2181233000' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:28:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:21.464+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19d802d170 con 0x7f19d81017a0 2024-04-25T00:28:21.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3400002599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:21.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:21 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2679351796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:28:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2181233000' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:28:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3400002599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:21 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2679351796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:28:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2181233000' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:28:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.380+0000 7f19d57fa640 1 -- 172.21.15.31:0/3348028446 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f19c80bc150 con 0x7f19d81017a0 2024-04-25T00:28:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f19c00a2b90 msgr2=0x7f19c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f19c00a2b90 0x7f19c00a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19d8198eb0 tx=0x7f19cc00a2d0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 msgr2=0x7f19d81979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d81979a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19c80314e0 tx=0x7f19c80042e0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 shutdown_connections 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d8103350 0x7f19d819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19d8102990 0x7f19d8197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f19c00a2b90 0x7f19c00a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 --2- 172.21.15.31:0/3348028446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81017a0 0x7f19d81979a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 >> 172.21.15.31:0/3348028446 conn(0x7f19d80fcd10 msgr2=0x7f19d8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:22.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 shutdown_connections 2024-04-25T00:28:22.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.382+0000 7f19de8b2640 1 -- 172.21.15.31:0/3348028446 wait complete. 2024-04-25T00:28:22.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:28:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.509+0000 7f7462f1b640 1 -- 172.21.15.31:0/2041082306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c102280 msgr2=0x7f745c104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.509+0000 7f7462f1b640 1 --2- 172.21.15.31:0/2041082306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c102280 0x7f745c104670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c02a410 comp rx=0 tx=0).stop 2024-04-25T00:28:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- 172.21.15.31:0/2041082306 shutdown_connections 2024-04-25T00:28:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 --2- 172.21.15.31:0/2041082306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c1074e0 0x7f745c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 --2- 172.21.15.31:0/2041082306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c104bb0 0x7f745c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 --2- 172.21.15.31:0/2041082306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c102280 0x7f745c104670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- 172.21.15.31:0/2041082306 >> 172.21.15.31:0/2041082306 conn(0x7f745c0fbc30 msgr2=0x7f745c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- 172.21.15.31:0/2041082306 shutdown_connections 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- 172.21.15.31:0/2041082306 wait complete. 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 Processor -- start 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- start start 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c102280 0x7f745c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c104bb0 0x7f745c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 0x7f745c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f745c10d2c0 con 0x7f745c1074e0 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f745c10d140 con 0x7f745c104bb0 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7462f1b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f745c10d440 con 0x7f745c102280 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f74614b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 0x7f745c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f74614b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 0x7f745c19f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41484/0 (socket says 172.21.15.31:41484) 2024-04-25T00:28:22.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f74614b8640 1 -- 172.21.15.31:0/85337791 learned_addr learned my addr 172.21.15.31:0/85337791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f7460cb7640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c102280 0x7f745c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.510+0000 7f745bfff640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c104bb0 0x7f745c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f74614b8640 1 -- 172.21.15.31:0/85337791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c102280 msgr2=0x7f745c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f74614b8640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c102280 0x7f745c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f74614b8640 1 -- 172.21.15.31:0/85337791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c104bb0 msgr2=0x7f745c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f74614b8640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c104bb0 0x7f745c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f74614b8640 1 -- 172.21.15.31:0/85337791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c19f6a0 con 0x7f745c1074e0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f74614b8640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 0x7f745c19f030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7454009fd0 tx=0x7f745400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454017070 con 0x7f745c1074e0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7454003e20 con 0x7f745c1074e0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c19f990 con 0x7f745c1074e0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454010460 con 0x7f745c1074e0 2024-04-25T00:28:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.511+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c19fee0 con 0x7f745c1074e0 2024-04-25T00:28:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.512+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c0fd720 con 0x7f745c1074e0 2024-04-25T00:28:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.513+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7454003970 con 0x7f745c1074e0 2024-04-25T00:28:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.514+0000 7f7459ffb640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440a2c10 0x7f74440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.514+0000 7f7460cb7640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440a2c10 0x7f74440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.515+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f74540cea40 con 0x7f745c1074e0 2024-04-25T00:28:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.515+0000 7f7460cb7640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440a2c10 0x7f74440a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f744c006010 tx=0x7f744c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:22.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.519+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f745409b680 con 0x7f745c1074e0 2024-04-25T00:28:22.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:22 smithi169 ceph-mon[33112]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:22.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:22 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3348028446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:28:22.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:22 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:22 smithi040 ceph-mon[32945]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:22.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:22 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3348028446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:28:22.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:22 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:22.704+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f745c062340 con 0x7f745c1074e0 2024-04-25T00:28:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:22 smithi031 ceph-mon[29167]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:22 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3348028446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:28:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:22 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.382+0000 7f7459ffb640 1 -- 172.21.15.31:0/85337791 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f745409f980 con 0x7f745c1074e0 2024-04-25T00:28:23.383 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440a2c10 msgr2=0x7f74440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440a2c10 0x7f74440a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f744c006010 tx=0x7f744c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 msgr2=0x7f745c19f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 0x7f745c19f030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7454009fd0 tx=0x7f745400b260 comp rx=0 tx=0).stop 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 shutdown_connections 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74440a2c10 0x7f74440a50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c1074e0 0x7f745c19f030 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c104bb0 0x7f745c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 --2- 172.21.15.31:0/85337791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c102280 0x7f745c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 >> 172.21.15.31:0/85337791 conn(0x7f745c0fbc30 msgr2=0x7f745c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 shutdown_connections 2024-04-25T00:28:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.385+0000 7f7462f1b640 1 -- 172.21.15.31:0/85337791 wait complete. 2024-04-25T00:28:23.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:28:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 -- 172.21.15.31:0/2929252011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 msgr2=0x7fab3c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 --2- 172.21.15.31:0/2929252011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c109be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009900 tx=0x7fab2c02a410 comp rx=0 tx=0).stop 2024-04-25T00:28:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 -- 172.21.15.31:0/2929252011 shutdown_connections 2024-04-25T00:28:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 --2- 172.21.15.31:0/2929252011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c109be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 --2- 172.21.15.31:0/2929252011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab3c102990 0x7fab3c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 --2- 172.21.15.31:0/2929252011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 0x7fab3c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 -- 172.21.15.31:0/2929252011 >> 172.21.15.31:0/2929252011 conn(0x7fab3c0fcd10 msgr2=0x7fab3c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 -- 172.21.15.31:0/2929252011 shutdown_connections 2024-04-25T00:28:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 -- 172.21.15.31:0/2929252011 wait complete. 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 Processor -- start 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.511+0000 7fab42a72640 1 -- start start 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 0x7fab3c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab3c102990 0x7fab3c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10d2d0 con 0x7fab3c103350 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10d150 con 0x7fab3c102990 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10d450 con 0x7fab3c1017a0 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49980/0 (socket says 172.21.15.31:49980) 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 -- 172.21.15.31:0/695128391 learned_addr learned my addr 172.21.15.31:0/695128391 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab3bfff640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab3c102990 0x7fab3c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4080e640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 0x7fab3c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 -- 172.21.15.31:0/695128391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 msgr2=0x7fab3c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 0x7fab3c197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 -- 172.21.15.31:0/695128391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab3c102990 msgr2=0x7fab3c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab3c102990 0x7fab3c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 -- 172.21.15.31:0/695128391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c19f5a0 con 0x7fab3c103350 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4080e640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 0x7fab3c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab4100f640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c19ef30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005940 tx=0x7fab2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c03f030 con 0x7fab3c103350 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c19f830 con 0x7fab3c103350 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab2c0318f0 con 0x7fab3c103350 2024-04-25T00:28:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.512+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c035070 con 0x7fab3c103350 2024-04-25T00:28:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.513+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c19fd20 con 0x7fab3c103350 2024-04-25T00:28:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.514+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c108ac0 con 0x7fab3c103350 2024-04-25T00:28:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.515+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab2c042030 con 0x7fab3c103350 2024-04-25T00:28:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.515+0000 7fab39ffb640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab240a2b90 0x7fab240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.516+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7fab2c0ef400 con 0x7fab3c103350 2024-04-25T00:28:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.516+0000 7fab4080e640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab240a2b90 0x7fab240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.516+0000 7fab4080e640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab240a2b90 0x7fab240a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab3c1027f0 tx=0x7fab34009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.520+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab2c0bc510 con 0x7fab3c103350 2024-04-25T00:28:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:23.751+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fab3c062340 con 0x7fab3c103350 2024-04-25T00:28:23.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3348028446' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:28:23.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:28:23.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:23.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:23.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:23 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/85337791' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3348028446' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:23 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/85337791' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:28:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3348028446' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:28:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:28:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:23.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:28:23.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:23 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/85337791' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:28:24.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.402+0000 7fab39ffb640 1 -- 172.21.15.31:0/695128391 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fab2c0bbe10 con 0x7fab3c103350 2024-04-25T00:28:24.403 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab240a2b90 msgr2=0x7fab240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab240a2b90 0x7fab240a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab3c1027f0 tx=0x7fab34009500 comp rx=0 tx=0).stop 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 msgr2=0x7fab3c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c19ef30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005940 tx=0x7fab2c0042f0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 shutdown_connections 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab240a2b90 0x7fab240a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c103350 0x7fab3c19ef30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab3c102990 0x7fab3c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 --2- 172.21.15.31:0/695128391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab3c1017a0 0x7fab3c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 >> 172.21.15.31:0/695128391 conn(0x7fab3c0fcd10 msgr2=0x7fab3c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 shutdown_connections 2024-04-25T00:28:24.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.406+0000 7fab42a72640 1 -- 172.21.15.31:0/695128391 wait complete. 2024-04-25T00:28:24.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:28:24.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.527+0000 7f625df1b640 1 -- 172.21.15.31:0/1938004144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62581006a0 msgr2=0x7f6258100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.527+0000 7f625df1b640 1 --2- 172.21.15.31:0/1938004144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62581006a0 0x7f6258100ac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6248009900 tx=0x7f624802a410 comp rx=0 tx=0).stop 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 -- 172.21.15.31:0/1938004144 shutdown_connections 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 --2- 172.21.15.31:0/1938004144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6258102250 0x7f6258108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 --2- 172.21.15.31:0/1938004144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6258101890 0x7f6258101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 --2- 172.21.15.31:0/1938004144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62581006a0 0x7f6258100ac0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 -- 172.21.15.31:0/1938004144 >> 172.21.15.31:0/1938004144 conn(0x7f62580fbc30 msgr2=0x7f62580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 -- 172.21.15.31:0/1938004144 shutdown_connections 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 -- 172.21.15.31:0/1938004144 wait complete. 2024-04-25T00:28:24.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 Processor -- start 2024-04-25T00:28:24.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.528+0000 7f625df1b640 1 -- start start 2024-04-25T00:28:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62581006a0 0x7f625806a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 0x7f625819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6258102250 0x7f625819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f625810c1d0 con 0x7f6258101890 2024-04-25T00:28:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f625810c050 con 0x7f62581006a0 2024-04-25T00:28:24.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f625810c350 con 0x7f6258102250 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 0x7f625819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 0x7f625819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49988/0 (socket says 172.21.15.31:49988) 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 -- 172.21.15.31:0/1934254041 learned_addr learned my addr 172.21.15.31:0/1934254041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f62577fe640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62581006a0 0x7f625806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6257fff640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6258102250 0x7f625819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 -- 172.21.15.31:0/1934254041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6258102250 msgr2=0x7f625819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6258102250 0x7f625819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 -- 172.21.15.31:0/1934254041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62581006a0 msgr2=0x7f625806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62581006a0 0x7f625806a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 -- 172.21.15.31:0/1934254041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625819f640 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6256ffd640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 0x7f625819c740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f624000d770 tx=0x7f624000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240017070 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62400054d0 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625819f930 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240004cf0 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.529+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625819fe80 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.530+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6258107b30 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.532+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6240004e90 con 0x7f6258101890 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.532+0000 7f6254ff9640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62440a2c10 0x7f62440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.532+0000 7f62577fe640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62440a2c10 0x7f62440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.532+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f62400ce8e0 con 0x7f6258101890 2024-04-25T00:28:24.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.533+0000 7f62577fe640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62440a2c10 0x7f62440a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6248004770 tx=0x7f6248000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:24.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.537+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f624009b990 con 0x7f6258101890 2024-04-25T00:28:24.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.722+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f625802d3c0 con 0x7f6258101890 2024-04-25T00:28:24.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.723+0000 7f6254ff9640 1 -- 172.21.15.31:0/1934254041 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22222 (secure 0 0 0) 0x7f624009b290 con 0x7f6258101890 2024-04-25T00:28:24.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:24.724 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:28:23.752632+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62440a2c10 msgr2=0x7f62440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62440a2c10 0x7f62440a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6248004770 tx=0x7f6248000bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 msgr2=0x7f625819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 0x7f625819c740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f624000d770 tx=0x7f624000dc30 comp rx=0 tx=0).stop 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 shutdown_connections 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6258102250 0x7f625819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62440a2c10 0x7f62440a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6258101890 0x7f625819c740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 --2- 172.21.15.31:0/1934254041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62581006a0 0x7f625806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 >> 172.21.15.31:0/1934254041 conn(0x7f62580fbc30 msgr2=0x7f6258104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 shutdown_connections 2024-04-25T00:28:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.725+0000 7f625df1b640 1 -- 172.21.15.31:0/1934254041 wait complete. 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: pgmap v289: 33 pgs: 16 creating+peering, 1 active+clean, 16 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/85337791' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/695128391' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:28:24.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:24 smithi031 ceph-mon[29167]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:28:24.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: pgmap v289: 33 pgs: 16 creating+peering, 1 active+clean, 16 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/85337791' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/695128391' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:28:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:24 smithi169 ceph-mon[33112]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:28:24.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/2261936126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 msgr2=0x7fb9a0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/2261936126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a0100ac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb998009900 tx=0x7fb99802a410 comp rx=0 tx=0).stop 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/2261936126 shutdown_connections 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/2261936126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0102250 0x7fb9a0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/2261936126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a0101890 0x7fb9a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/2261936126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a0100ac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/2261936126 >> 172.21.15.31:0/2261936126 conn(0x7fb9a00fbc30 msgr2=0x7fb9a00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/2261936126 shutdown_connections 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.838+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/2261936126 wait complete. 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 Processor -- start 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 -- start start 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a010acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0101890 0x7fb9a0109360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a0102250 0x7fb9a01098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a010d370 con 0x7fb9a01006a0 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a010d1f0 con 0x7fb9a0101890 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9a4fd5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a010d4f0 con 0x7fb9a0102250 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a010acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a010acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49998/0 (socket says 172.21.15.31:49998) 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 -- 172.21.15.31:0/4159107697 learned_addr learned my addr 172.21.15.31:0/4159107697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99ed9d640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a0102250 0x7fb9a01098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99dd9b640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0101890 0x7fb9a0109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 -- 172.21.15.31:0/4159107697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a0102250 msgr2=0x7fb9a01098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a0102250 0x7fb9a01098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 -- 172.21.15.31:0/4159107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0101890 msgr2=0x7fb9a0109360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0101890 0x7fb9a0109360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 -- 172.21.15.31:0/4159107697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a010a160 con 0x7fb9a01006a0 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99dd9b640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0101890 0x7fb9a0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb99e59c640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a010acd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9980314e0 tx=0x7fb998003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.839+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99803f070 con 0x7fb9a01006a0 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.840+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb998031e40 con 0x7fb9a01006a0 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.840+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01a1b30 con 0x7fb9a01006a0 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.840+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035030 con 0x7fb9a01006a0 2024-04-25T00:28:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.840+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a2040 con 0x7fb9a01006a0 2024-04-25T00:28:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.841+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a0107a90 con 0x7fb9a01006a0 2024-04-25T00:28:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.842+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb998042070 con 0x7fb9a01006a0 2024-04-25T00:28:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.842+0000 7fb9977fe640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb98c0a2c10 0x7fb98c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.843+0000 7fb99dd9b640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb98c0a2c10 0x7fb98c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.843+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb9980ef770 con 0x7fb9a01006a0 2024-04-25T00:28:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.843+0000 7fb99dd9b640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb98c0a2c10 0x7fb98c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9a010aa40 tx=0x7fb988005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:24.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:24.847+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb9980bc820 con 0x7fb9a01006a0 2024-04-25T00:28:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: pgmap v289: 33 pgs: 16 creating+peering, 1 active+clean, 16 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/85337791' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:28:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:28:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/695128391' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:28:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T00:28:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:28:24.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:24 smithi040 ceph-mon[32945]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:28:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.034+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9a002d3c0 con 0x7fb9a01006a0 2024-04-25T00:28:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.035+0000 7fb9977fe640 1 -- 172.21.15.31:0/4159107697 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22222 (secure 0 0 0) 0x7fb9980bc120 con 0x7fb9a01006a0 2024-04-25T00:28:25.036 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:25.036 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:28:23.752632+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:28:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb98c0a2c10 msgr2=0x7fb98c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb98c0a2c10 0x7fb98c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9a010aa40 tx=0x7fb988005e50 comp rx=0 tx=0).stop 2024-04-25T00:28:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 msgr2=0x7fb9a010acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a010acd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9980314e0 tx=0x7fb998003c80 comp rx=0 tx=0).stop 2024-04-25T00:28:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 shutdown_connections 2024-04-25T00:28:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a0102250 0x7fb9a01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.037+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a0101890 0x7fb9a0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.038+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb98c0a2c10 0x7fb98c0a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.038+0000 7fb9a4fd5640 1 --2- 172.21.15.31:0/4159107697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a01006a0 0x7fb9a010acd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.038+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 >> 172.21.15.31:0/4159107697 conn(0x7fb9a00fbc30 msgr2=0x7fb9a0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:25.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.038+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 shutdown_connections 2024-04-25T00:28:25.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.038+0000 7fb9a4fd5640 1 -- 172.21.15.31:0/4159107697 wait complete. 2024-04-25T00:28:25.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:28:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 -- 172.21.15.31:0/1605801952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0102280 msgr2=0x7fdca0104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 --2- 172.21.15.31:0/1605801952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0102280 0x7fdca0104670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009900 tx=0x7fdc9c02a410 comp rx=0 tx=0).stop 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 -- 172.21.15.31:0/1605801952 shutdown_connections 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 --2- 172.21.15.31:0/1605801952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca01074e0 0x7fdca01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 --2- 172.21.15.31:0/1605801952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca0104bb0 0x7fdca0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 --2- 172.21.15.31:0/1605801952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0102280 0x7fdca0104670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 -- 172.21.15.31:0/1605801952 >> 172.21.15.31:0/1605801952 conn(0x7fdca00fbc30 msgr2=0x7fdca00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 -- 172.21.15.31:0/1605801952 shutdown_connections 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.154+0000 7fdca829e640 1 -- 172.21.15.31:0/1605801952 wait complete. 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 Processor -- start 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 -- start start 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca0102280 0x7fdca0101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 0x7fdca0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca01074e0 0x7fdca0100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010d2c0 con 0x7fdca0104bb0 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010d140 con 0x7fdca0102280 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca829e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010d440 con 0x7fdca01074e0 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 0x7fdca0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 0x7fdca0100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50018/0 (socket says 172.21.15.31:50018) 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 -- 172.21.15.31:0/2876269680 learned_addr learned my addr 172.21.15.31:0/2876269680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca603a640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca0102280 0x7fdca0101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca683b640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca01074e0 0x7fdca0100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 -- 172.21.15.31:0/2876269680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca01074e0 msgr2=0x7fdca0100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca01074e0 0x7fdca0100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 -- 172.21.15.31:0/2876269680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca0102280 msgr2=0x7fdca0101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca0102280 0x7fdca0101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 -- 172.21.15.31:0/2876269680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca0101000 con 0x7fdca0104bb0 2024-04-25T00:28:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.155+0000 7fdca5839640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 0x7fdca0100230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000d770 tx=0x7fdc9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.156+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90017070 con 0x7fdca0104bb0 2024-04-25T00:28:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.156+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca01a1b30 con 0x7fdca0104bb0 2024-04-25T00:28:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.156+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc90009c30 con 0x7fdca0104bb0 2024-04-25T00:28:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.156+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000f620 con 0x7fdca0104bb0 2024-04-25T00:28:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.156+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a2080 con 0x7fdca0104bb0 2024-04-25T00:28:25.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.157+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca00fd720 con 0x7fdca0104bb0 2024-04-25T00:28:25.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.158+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc9001a070 con 0x7fdca0104bb0 2024-04-25T00:28:25.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.159+0000 7fdc977fe640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc8c0a2b90 0x7fdc8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.159+0000 7fdca603a640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc8c0a2b90 0x7fdc8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.159+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdc900ce920 con 0x7fdca0104bb0 2024-04-25T00:28:25.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.160+0000 7fdca603a640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc8c0a2b90 0x7fdc8c0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.165+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc90009de0 con 0x7fdca0104bb0 2024-04-25T00:28:25.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.390+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdca002d370 con 0x7fdca0104bb0 2024-04-25T00:28:25.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.406+0000 7fdc977fe640 1 -- 172.21.15.31:0/2876269680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fdc9009b380 con 0x7fdca0104bb0 2024-04-25T00:28:25.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc8c0a2b90 msgr2=0x7fdc8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc8c0a2b90 0x7fdc8c0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 msgr2=0x7fdca0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 0x7fdca0100230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000d770 tx=0x7fdc9000dc30 comp rx=0 tx=0).stop 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 shutdown_connections 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca01074e0 0x7fdca0100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc8c0a2b90 0x7fdc8c0a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca0104bb0 0x7fdca0100230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 --2- 172.21.15.31:0/2876269680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca0102280 0x7fdca0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.407+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 >> 172.21.15.31:0/2876269680 conn(0x7fdca00fbc30 msgr2=0x7fdca0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.408+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 shutdown_connections 2024-04-25T00:28:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.408+0000 7fdca829e640 1 -- 172.21.15.31:0/2876269680 wait complete. 2024-04-25T00:28:25.421 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T00:28:25.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/1737201482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 msgr2=0x7f1d24101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/1737201482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24101cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009900 tx=0x7f1d1002a410 comp rx=0 tx=0).stop 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/1737201482 shutdown_connections 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/1737201482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 0x7f1d24108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/1737201482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24101cf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/1737201482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 0x7f1d24100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/1737201482 >> 172.21.15.31:0/1737201482 conn(0x7f1d240fbbf0 msgr2=0x7f1d240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/1737201482 shutdown_connections 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/1737201482 wait complete. 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.525+0000 7f1d2ac3f640 1 Processor -- start 2024-04-25T00:28:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 -- start start 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 0x7f1d24104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 0x7f1d24103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410c1b0 con 0x7f1d24101870 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410c030 con 0x7f1d24100680 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d2ac3f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410c330 con 0x7f1d24102230 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50044/0 (socket says 172.21.15.31:50044) 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 -- 172.21.15.31:0/2555976666 learned_addr learned my addr 172.21.15.31:0/2555976666 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d291dc640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 0x7f1d24103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d289db640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 0x7f1d24104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 -- 172.21.15.31:0/2555976666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 msgr2=0x7f1d24103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 0x7f1d24103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 -- 172.21.15.31:0/2555976666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 msgr2=0x7f1d24104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 0x7f1d24104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 -- 172.21.15.31:0/2555976666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d24103a20 con 0x7f1d24101870 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d291dc640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 0x7f1d24103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d289db640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 0x7f1d24104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d23fff640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24102c50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005940 tx=0x7f1d100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.526+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1003f030 con 0x7f1d24101870 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.527+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d241a1ac0 con 0x7f1d24101870 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.527+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d10003cb0 con 0x7f1d24101870 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.527+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10035070 con 0x7f1d24101870 2024-04-25T00:28:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.527+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a2060 con 0x7f1d24101870 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.528+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d241a2230 con 0x7f1d24101870 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.529+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d10042030 con 0x7f1d24101870 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.529+0000 7f1d21ffb640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d140a2b90 0x7f1d140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.530+0000 7f1d289db640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d140a2b90 0x7f1d140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.530+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f1d100ef550 con 0x7f1d24101870 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.530+0000 7f1d289db640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d140a2b90 0x7f1d140a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d180058c0 tx=0x7f1d18005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.534+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d100bc560 con 0x7f1d24101870 2024-04-25T00:28:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:25.757+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f1d241a25c0 con 0x7f1d24101870 2024-04-25T00:28:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/695128391' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:28:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T00:28:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T00:28:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1934254041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:25.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4159107697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:25.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:28:25.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:25 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2876269680' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:28:25.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/695128391' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:28:25.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T00:28:25.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T00:28:25.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1934254041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:25.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4159107697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:25.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:28:25.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:25 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2876269680' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/695128391' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1934254041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4159107697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:28:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:25 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2876269680' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:28:26.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.415+0000 7f1d21ffb640 1 -- 172.21.15.31:0/2555976666 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f1d100bbe60 con 0x7f1d24101870 2024-04-25T00:28:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d140a2b90 msgr2=0x7f1d140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d140a2b90 0x7f1d140a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d180058c0 tx=0x7f1d18005850 comp rx=0 tx=0).stop 2024-04-25T00:28:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 msgr2=0x7f1d24102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24102c50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005940 tx=0x7f1d100042f0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 shutdown_connections 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d24102230 0x7f1d24103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d140a2b90 0x7f1d140a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d24101870 0x7f1d24102c50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 --2- 172.21.15.31:0/2555976666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24100680 0x7f1d24104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 >> 172.21.15.31:0/2555976666 conn(0x7f1d240fbbf0 msgr2=0x7f1d24068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 shutdown_connections 2024-04-25T00:28:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.420+0000 7f1d2ac3f640 1 -- 172.21.15.31:0/2555976666 wait complete. 2024-04-25T00:28:26.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T00:28:26.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3464397640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38100970 msgr2=0x7f7a38100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:26.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3464397640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38100970 0x7f7a38100dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009900 tx=0x7f7a2802a410 comp rx=0 tx=0).stop 2024-04-25T00:28:26.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3464397640 shutdown_connections 2024-04-25T00:28:26.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3464397640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38101310 0x7f7a38101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3464397640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38100970 0x7f7a38100dd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3464397640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a38100010 0x7f7a38100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3464397640 >> 172.21.15.31:0/3464397640 conn(0x7f7a380fbc30 msgr2=0x7f7a380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3464397640 shutdown_connections 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.547+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3464397640 wait complete. 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 Processor -- start 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 -- start start 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a38100010 0x7f7a38199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38100970 0x7f7a3819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 0x7f7a381a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810f560 con 0x7f7a38101310 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810f3e0 con 0x7f7a38100970 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3fb90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810f6e0 con 0x7f7a38100010 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 0x7f7a381a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 0x7f7a381a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50072/0 (socket says 172.21.15.31:50072) 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3d92c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a38100010 0x7f7a38199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 -- 172.21.15.31:0/3639287883 learned_addr learned my addr 172.21.15.31:0/3639287883 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3d12b640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38100970 0x7f7a3819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 -- 172.21.15.31:0/3639287883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a38100010 msgr2=0x7f7a38199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a38100010 0x7f7a38199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 -- 172.21.15.31:0/3639287883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38100970 msgr2=0x7f7a3819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38100970 0x7f7a3819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3e12d640 1 -- 172.21.15.31:0/3639287883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a381a1920 con 0x7f7a38101310 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.548+0000 7f7a3d12b640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38100970 0x7f7a3819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.549+0000 7f7a3e12d640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 0x7f7a381a12b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b970 tx=0x7f7a3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.549+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30019070 con 0x7f7a38101310 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.549+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a30014440 con 0x7f7a38101310 2024-04-25T00:28:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.549+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a381a1c10 con 0x7f7a38101310 2024-04-25T00:28:26.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.549+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300075d0 con 0x7f7a38101310 2024-04-25T00:28:26.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.549+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381a2130 con 0x7f7a38101310 2024-04-25T00:28:26.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.550+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3810ad60 con 0x7f7a38101310 2024-04-25T00:28:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.551+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a3001c070 con 0x7f7a38101310 2024-04-25T00:28:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.551+0000 7f7a2effd640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7a240a2c10 0x7f7a240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.552+0000 7f7a3d92c640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7a240a2c10 0x7f7a240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.552+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7a300ce4a0 con 0x7f7a38101310 2024-04-25T00:28:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.552+0000 7f7a3d92c640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7a240a2c10 0x7f7a240a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400bbe0 tx=0x7f7a3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.556+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7a3009ae60 con 0x7f7a38101310 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: pgmap v293: 97 pgs: 29 creating+peering, 15 active+clean, 53 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2876269680' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] up:active 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2555976666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:28:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:26 smithi031 ceph-mon[29167]: Cluster is now healthy 2024-04-25T00:28:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:26.781+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f7a380623e0 con 0x7f7a38101310 2024-04-25T00:28:26.796 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: pgmap v293: 97 pgs: 29 creating+peering, 15 active+clean, 53 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:26.797 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2876269680' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:28:26.797 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] up:active 2024-04-25T00:28:26.797 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T00:28:26.797 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2555976666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T00:28:26.797 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:28:26.798 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:26 smithi040 ceph-mon[32945]: Cluster is now healthy 2024-04-25T00:28:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: pgmap v293: 97 pgs: 29 creating+peering, 15 active+clean, 53 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:28:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2876269680' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:28:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] up:active 2024-04-25T00:28:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T00:28:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2555976666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T00:28:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:28:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:26 smithi169 ceph-mon[33112]: Cluster is now healthy 2024-04-25T00:28:27.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.426+0000 7f7a2effd640 1 -- 172.21.15.31:0/3639287883 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f7a3009f160 con 0x7f7a38101310 2024-04-25T00:28:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7a240a2c10 msgr2=0x7f7a240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7a240a2c10 0x7f7a240a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400bbe0 tx=0x7f7a3400b390 comp rx=0 tx=0).stop 2024-04-25T00:28:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 msgr2=0x7f7a381a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 0x7f7a381a12b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b970 tx=0x7f7a3000be30 comp rx=0 tx=0).stop 2024-04-25T00:28:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 shutdown_connections 2024-04-25T00:28:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7a240a2c10 0x7f7a240a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a38101310 0x7f7a381a12b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a38100970 0x7f7a3819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 --2- 172.21.15.31:0/3639287883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a38100010 0x7f7a38199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 >> 172.21.15.31:0/3639287883 conn(0x7f7a380fbc30 msgr2=0x7f7a380fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 shutdown_connections 2024-04-25T00:28:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.429+0000 7f7a3fb90640 1 -- 172.21.15.31:0/3639287883 wait complete. 2024-04-25T00:28:27.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T00:28:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2775434982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14107460 msgr2=0x7f8f141098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2775434982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14107460 0x7f8f141098b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009ca0 tx=0x7f8f0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2775434982 shutdown_connections 2024-04-25T00:28:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2775434982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14107460 0x7f8f141098b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2775434982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f14104b30 0x7f8f14106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2775434982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f141045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.557+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2775434982 >> 172.21.15.31:0/2775434982 conn(0x7f8f140fbbf0 msgr2=0x7f8f140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2775434982 shutdown_connections 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2775434982 wait complete. 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 Processor -- start 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 -- start start 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f14101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14104b30 0x7f8f141001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f14107460 0x7f8f141006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410d2d0 con 0x7f8f14104b30 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410d150 con 0x7f8f14107460 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1c3c8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1410d450 con 0x7f8f14102200 2024-04-25T00:28:27.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f19963640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14104b30 0x7f8f141001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1a164640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f14101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1a164640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f14101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:46150/0 (socket says 172.21.15.31:46150) 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f1a164640 1 -- 172.21.15.31:0/2634966171 learned_addr learned my addr 172.21.15.31:0/2634966171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.558+0000 7f8f19963640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14104b30 0x7f8f141001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50090/0 (socket says 172.21.15.31:50090) 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a965640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f14107460 0x7f8f141006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a164640 1 -- 172.21.15.31:0/2634966171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f14107460 msgr2=0x7f8f141006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a164640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f14107460 0x7f8f141006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a164640 1 -- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14104b30 msgr2=0x7f8f141001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a164640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14104b30 0x7f8f141001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a164640 1 -- 172.21.15.31:0/2634966171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f14100f80 con 0x7f8f14102200 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1a164640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f14101b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400e9f0 tx=0x7f8f0400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400cd80 con 0x7f8f14102200 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f141a1b00 con 0x7f8f14102200 2024-04-25T00:28:27.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f040054d0 con 0x7f8f14102200 2024-04-25T00:28:27.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04010490 con 0x7f8f14102200 2024-04-25T00:28:27.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.559+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a2050 con 0x7f8f14102200 2024-04-25T00:28:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.560+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f140fd6e0 con 0x7f8f14102200 2024-04-25T00:28:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.561+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f04005020 con 0x7f8f14102200 2024-04-25T00:28:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.562+0000 7f8f0b7fe640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f000a2b90 0x7f8f000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.562+0000 7f8f19963640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f000a2b90 0x7f8f000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.562+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8f040cf350 con 0x7f8f14102200 2024-04-25T00:28:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.562+0000 7f8f19963640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f000a2b90 0x7f8f000a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f141018e0 tx=0x7f8f10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.565+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f0409c360 con 0x7f8f14102200 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2555976666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T00:28:27.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:27 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3639287883' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:28:27.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:27.790+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8f1402d210 con 0x7f8f14102200 2024-04-25T00:28:27.809 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:28:27.809 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2555976666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T00:28:27.809 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:28:27.809 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T00:28:27.809 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T00:28:27.809 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T00:28:27.810 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:27 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3639287883' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:28:27.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:28:27.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2555976666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T00:28:27.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:28:27.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T00:28:27.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T00:28:27.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T00:28:27.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:27 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3639287883' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:28:28.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.157+0000 7f8f0b7fe640 1 -- 172.21.15.31:0/2634966171 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f8f0409bc60 con 0x7f8f14102200 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f000a2b90 msgr2=0x7f8f000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f000a2b90 0x7f8f000a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f141018e0 tx=0x7f8f10005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 msgr2=0x7f8f14101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f14101b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400e9f0 tx=0x7f8f0400edf0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 shutdown_connections 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f14107460 0x7f8f141006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f000a2b90 0x7f8f000a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f14104b30 0x7f8f141001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 --2- 172.21.15.31:0/2634966171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14102200 0x7f8f14101b70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.159+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 >> 172.21.15.31:0/2634966171 conn(0x7f8f140fbbf0 msgr2=0x7f8f14102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.160+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 shutdown_connections 2024-04-25T00:28:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.160+0000 7f8f1c3c8640 1 -- 172.21.15.31:0/2634966171 wait complete. 2024-04-25T00:28:28.177 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-25T00:28:28.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 -- 172.21.15.31:0/2354726219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 msgr2=0x7f9fe4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 --2- 172.21.15.31:0/2354726219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe4100aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009900 tx=0x7f9fe002a410 comp rx=0 tx=0).stop 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 -- 172.21.15.31:0/2354726219 shutdown_connections 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 --2- 172.21.15.31:0/2354726219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 0x7f9fe4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 --2- 172.21.15.31:0/2354726219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fe4101870 0x7f9fe4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 --2- 172.21.15.31:0/2354726219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe4100aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 -- 172.21.15.31:0/2354726219 >> 172.21.15.31:0/2354726219 conn(0x7f9fe40fbbf0 msgr2=0x7f9fe40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 -- 172.21.15.31:0/2354726219 shutdown_connections 2024-04-25T00:28:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.283+0000 7f9feab29640 1 -- 172.21.15.31:0/2354726219 wait complete. 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 Processor -- start 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 -- start start 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fe4101870 0x7f9fe4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 0x7f9fe419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe410d2d0 con 0x7f9fe4100680 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe410d150 con 0x7f9fe4101870 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9feab29640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe410d450 con 0x7f9fe4102230 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe41979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50118/0 (socket says 172.21.15.31:50118) 2024-04-25T00:28:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 -- 172.21.15.31:0/3935337127 learned_addr learned my addr 172.21.15.31:0/3935337127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe90c6640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 0x7f9fe419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fdbfff640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fe4101870 0x7f9fe4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 -- 172.21.15.31:0/3935337127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 msgr2=0x7f9fe419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 0x7f9fe419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 -- 172.21.15.31:0/3935337127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fe4101870 msgr2=0x7f9fe4197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fe4101870 0x7f9fe4197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 -- 172.21.15.31:0/3935337127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe419f5e0 con 0x7f9fe4100680 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe90c6640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 0x7f9fe419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.284+0000 7f9fe88c5640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe41979c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00314e0 tx=0x7f9fe0031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.285+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe003f030 con 0x7f9fe4100680 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.285+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fe00044b0 con 0x7f9fe4100680 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.285+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe419f870 con 0x7f9fe4100680 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.285+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0035070 con 0x7f9fe4100680 2024-04-25T00:28:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.285+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe419fdb0 con 0x7f9fe4100680 2024-04-25T00:28:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.286+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4107a20 con 0x7f9fe4100680 2024-04-25T00:28:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.287+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9fe0042030 con 0x7f9fe4100680 2024-04-25T00:28:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.287+0000 7f9fd9ffb640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9fcc0a2c10 0x7f9fcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.288+0000 7f9fdbfff640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9fcc0a2c10 0x7f9fcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.288+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f9fe00ef480 con 0x7f9fe4100680 2024-04-25T00:28:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.288+0000 7f9fdbfff640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9fcc0a2c10 0x7f9fcc0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005fd0 tx=0x7f9fd4005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:28.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.292+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9fe00bc490 con 0x7f9fe4100680 2024-04-25T00:28:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.505+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9fe4061b80 con 0x7f9fcc0a2c10 2024-04-25T00:28:28.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.528+0000 7f9fd9ffb640 1 -- 172.21.15.31:0/3935337127 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9fe4061b80 con 0x7f9fcc0a2c10 2024-04-25T00:28:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.530+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9fcc0a2c10 msgr2=0x7f9fcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.530+0000 7f9feab29640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9fcc0a2c10 0x7f9fcc0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005fd0 tx=0x7f9fd4005c70 comp rx=0 tx=0).stop 2024-04-25T00:28:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.530+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 msgr2=0x7f9fe41979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.530+0000 7f9feab29640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe41979c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00314e0 tx=0x7f9fe0031a30 comp rx=0 tx=0).stop 2024-04-25T00:28:28.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 shutdown_connections 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe4102230 0x7f9fe419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fe4101870 0x7f9fe4197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9fcc0a2c10 0x7f9fcc0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 --2- 172.21.15.31:0/3935337127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe4100680 0x7f9fe41979c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 >> 172.21.15.31:0/3935337127 conn(0x7f9fe40fbbf0 msgr2=0x7f9fe4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 shutdown_connections 2024-04-25T00:28:28.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.531+0000 7f9feab29640 1 -- 172.21.15.31:0/3935337127 wait complete. 2024-04-25T00:28:28.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 -- 172.21.15.31:0/2907840601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398101890 msgr2=0x7f7398101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 --2- 172.21.15.31:0/2907840601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398101890 0x7f7398101d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7380009900 tx=0x7f738002a410 comp rx=0 tx=0).stop 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 -- 172.21.15.31:0/2907840601 shutdown_connections 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 --2- 172.21.15.31:0/2907840601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398102250 0x7f7398108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 --2- 172.21.15.31:0/2907840601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398101890 0x7f7398101d10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 --2- 172.21.15.31:0/2907840601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73981006a0 0x7f7398100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 -- 172.21.15.31:0/2907840601 >> 172.21.15.31:0/2907840601 conn(0x7f73980fbc30 msgr2=0x7f73980fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 -- 172.21.15.31:0/2907840601 shutdown_connections 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.648+0000 7f739cdb7640 1 -- 172.21.15.31:0/2907840601 wait complete. 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 Processor -- start 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 -- start start 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73981006a0 0x7f739806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398101890 0x7f739819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 0x7f739819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f739810c200 con 0x7f7398102250 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f739810c080 con 0x7f7398101890 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739cdb7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f739810c380 con 0x7f73981006a0 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 0x7f739819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 0x7f739819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50162/0 (socket says 172.21.15.31:50162) 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 -- 172.21.15.31:0/3788542031 learned_addr learned my addr 172.21.15.31:0/3788542031 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f739659c640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73981006a0 0x7f739806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7395d9b640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398101890 0x7f739819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 -- 172.21.15.31:0/3788542031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73981006a0 msgr2=0x7f739806a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73981006a0 0x7f739806a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 -- 172.21.15.31:0/3788542031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398101890 msgr2=0x7f739819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398101890 0x7f739819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 -- 172.21.15.31:0/3788542031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739819f650 con 0x7f7398102250 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7395d9b640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398101890 0x7f739819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.649+0000 7f7396d9d640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 0x7f739819ed90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f739000c970 tx=0x7f739000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.650+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390007bb0 con 0x7f7398102250 2024-04-25T00:28:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.650+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739819f940 con 0x7f7398102250 2024-04-25T00:28:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.650+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7390007d50 con 0x7f7398102250 2024-04-25T00:28:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.650+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390004e10 con 0x7f7398102250 2024-04-25T00:28:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.650+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739819fe40 con 0x7f7398102250 2024-04-25T00:28:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.651+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7398107a30 con 0x7f7398102250 2024-04-25T00:28:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.652+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7390005020 con 0x7f7398102250 2024-04-25T00:28:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.652+0000 7f738f7fe640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73840a2b90 0x7f73840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.652+0000 7f739659c640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73840a2b90 0x7f73840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.652+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f73900cebc0 con 0x7f7398102250 2024-04-25T00:28:28.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.653+0000 7f739659c640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73840a2b90 0x7f73840a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f73981016f0 tx=0x7f7388009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.657+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7390014030 con 0x7f7398102250 2024-04-25T00:28:28.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:28:28.510+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:28:28.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:28:28.510+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:28:28.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:28:28.510+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:28:28.510+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:28:28.510+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: pgmap v294: 97 pgs: 29 creating+peering, 42 active+clean, 26 unknown; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3639287883' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] up:active 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T00:28:28.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2634966171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] up:active 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: daemon mds.j assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=j=up:creating} 3 up:standby-replay 5 up:standby 2024-04-25T00:28:28.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:28 smithi031 ceph-mon[29167]: daemon mds.j is now active in filesystem cephfs as rank 3 2024-04-25T00:28:28.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.871+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7398061b80 con 0x7f73840a2b90 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: pgmap v294: 97 pgs: 29 creating+peering, 42 active+clean, 26 unknown; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3639287883' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] up:active 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2634966171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] up:active 2024-04-25T00:28:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: daemon mds.j assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T00:28:28.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:28:28.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=j=up:creating} 3 up:standby-replay 5 up:standby 2024-04-25T00:28:28.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:28 smithi169 ceph-mon[33112]: daemon mds.j is now active in filesystem cephfs as rank 3 2024-04-25T00:28:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.893+0000 7f738f7fe640 1 -- 172.21.15.31:0/3788542031 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7398061b80 con 0x7f73840a2b90 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: pgmap v294: 97 pgs: 29 creating+peering, 42 active+clean, 26 unknown; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3639287883' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] up:active 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2634966171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:28:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:28:28.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] up:active 2024-04-25T00:28:28.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: daemon mds.j assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T00:28:28.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:28:28.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=j=up:creating} 3 up:standby-replay 5 up:standby 2024-04-25T00:28:28.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:28 smithi040 ceph-mon[32945]: daemon mds.j is now active in filesystem cephfs as rank 3 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73840a2b90 msgr2=0x7f73840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73840a2b90 0x7f73840a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f73981016f0 tx=0x7f7388009500 comp rx=0 tx=0).stop 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 msgr2=0x7f739819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 0x7f739819ed90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f739000c970 tx=0x7f739000ce30 comp rx=0 tx=0).stop 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 shutdown_connections 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73840a2b90 0x7f73840a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7398102250 0x7f739819ed90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7398101890 0x7f739819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 --2- 172.21.15.31:0/3788542031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73981006a0 0x7f739806a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 >> 172.21.15.31:0/3788542031 conn(0x7f73980fbc30 msgr2=0x7f7398104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 shutdown_connections 2024-04-25T00:28:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:28.896+0000 7f739cdb7640 1 -- 172.21.15.31:0/3788542031 wait complete. 2024-04-25T00:28:28.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 -- 172.21.15.31:0/1414990544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 msgr2=0x7faf90102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 --2- 172.21.15.31:0/1414990544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90102eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009930 tx=0x7faf7c02a420 comp rx=0 tx=0).stop 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 -- 172.21.15.31:0/1414990544 shutdown_connections 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 --2- 172.21.15.31:0/1414990544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf901033f0 0x7faf90109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 --2- 172.21.15.31:0/1414990544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90102eb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 --2- 172.21.15.31:0/1414990544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ffa60 0x7faf900ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 -- 172.21.15.31:0/1414990544 >> 172.21.15.31:0/1414990544 conn(0x7faf900fb370 msgr2=0x7faf900fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 -- 172.21.15.31:0/1414990544 shutdown_connections 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.013+0000 7faf97095640 1 -- 172.21.15.31:0/1414990544 wait complete. 2024-04-25T00:28:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 Processor -- start 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 -- start start 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ffa60 0x7faf901979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf901033f0 0x7faf9019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faf9010d2f0 con 0x7faf90102a30 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faf9010d170 con 0x7faf900ffa60 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf97095640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf9010d470 con 0x7faf901033f0 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50176/0 (socket says 172.21.15.31:50176) 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 -- 172.21.15.31:0/3463478996 learned_addr learned my addr 172.21.15.31:0/3463478996 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf95632640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf901033f0 0x7faf9019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf94e31640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ffa60 0x7faf901979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 -- 172.21.15.31:0/3463478996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf901033f0 msgr2=0x7faf9019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf901033f0 0x7faf9019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 -- 172.21.15.31:0/3463478996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ffa60 msgr2=0x7faf901979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ffa60 0x7faf901979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.014+0000 7faf8ffff640 1 -- 172.21.15.31:0/3463478996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9019f5f0 con 0x7faf90102a30 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.015+0000 7faf8ffff640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90197f10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf7c004770 tx=0x7faf7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.015+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c03f030 con 0x7faf90102a30 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.015+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf7c003aa0 con 0x7faf90102a30 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.015+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9019f880 con 0x7faf90102a30 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.015+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c035070 con 0x7faf90102a30 2024-04-25T00:28:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.015+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9019fdc0 con 0x7faf90102a30 2024-04-25T00:28:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.016+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf90108be0 con 0x7faf90102a30 2024-04-25T00:28:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.017+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf7c042030 con 0x7faf90102a30 2024-04-25T00:28:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.017+0000 7faf8dffb640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faf800a2c10 0x7faf800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.018+0000 7faf94e31640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faf800a2c10 0x7faf800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.018+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7faf7c0ef640 con 0x7faf90102a30 2024-04-25T00:28:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.018+0000 7faf94e31640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faf800a2c10 0x7faf800a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf84005db0 tx=0x7faf84005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:29.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.022+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faf7c0bc6f0 con 0x7faf90102a30 2024-04-25T00:28:29.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.240+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7faf90061cd0 con 0x7faf800a2c10 2024-04-25T00:28:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.245+0000 7faf8dffb640 1 -- 172.21.15.31:0/3463478996 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faf90061cd0 con 0x7faf800a2c10 2024-04-25T00:28:29.246 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/bf15116c-d217-4ec3-a264-717aa9c7473f 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faf800a2c10 msgr2=0x7faf800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faf800a2c10 0x7faf800a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf84005db0 tx=0x7faf84005d40 comp rx=0 tx=0).stop 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 msgr2=0x7faf90197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90197f10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf7c004770 tx=0x7faf7c004270 comp rx=0 tx=0).stop 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 shutdown_connections 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf901033f0 0x7faf9019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faf800a2c10 0x7faf800a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf90102a30 0x7faf90197f10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 --2- 172.21.15.31:0/3463478996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ffa60 0x7faf901979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 >> 172.21.15.31:0/3463478996 conn(0x7faf900fb370 msgr2=0x7faf90105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:29.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 shutdown_connections 2024-04-25T00:28:29.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.247+0000 7faf97095640 1 -- 172.21.15.31:0/3463478996 wait complete. 2024-04-25T00:28:29.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T00:28:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 -- 172.21.15.31:0/280899228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1441006a0 msgr2=0x7fd144100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 --2- 172.21.15.31:0/280899228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1441006a0 0x7fd144100ac0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd134009900 tx=0x7fd13402a410 comp rx=0 tx=0).stop 2024-04-25T00:28:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 -- 172.21.15.31:0/280899228 shutdown_connections 2024-04-25T00:28:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 --2- 172.21.15.31:0/280899228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144102250 0x7fd144108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 --2- 172.21.15.31:0/280899228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd144101890 0x7fd144101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 --2- 172.21.15.31:0/280899228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1441006a0 0x7fd144100ac0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 -- 172.21.15.31:0/280899228 >> 172.21.15.31:0/280899228 conn(0x7fd1440fbc30 msgr2=0x7fd1440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 -- 172.21.15.31:0/280899228 shutdown_connections 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.365+0000 7fd148d1b640 1 -- 172.21.15.31:0/280899228 wait complete. 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 Processor -- start 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 -- start start 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1441006a0 0x7fd14406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144101890 0x7fd14419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 0x7fd14419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c1d0 con 0x7fd144102250 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c050 con 0x7fd1441006a0 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd148d1b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c350 con 0x7fd144101890 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd142d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 0x7fd14419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd142d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 0x7fd14419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50194/0 (socket says 172.21.15.31:50194) 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd142d9d640 1 -- 172.21.15.31:0/4078607223 learned_addr learned my addr 172.21.15.31:0/4078607223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd13bfff640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144101890 0x7fd14419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.366+0000 7fd14259c640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1441006a0 0x7fd14406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd142d9d640 1 -- 172.21.15.31:0/4078607223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144101890 msgr2=0x7fd14419c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd142d9d640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144101890 0x7fd14419c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd142d9d640 1 -- 172.21.15.31:0/4078607223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1441006a0 msgr2=0x7fd14406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd142d9d640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1441006a0 0x7fd14406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd142d9d640 1 -- 172.21.15.31:0/4078607223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14419f6a0 con 0x7fd144102250 2024-04-25T00:28:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd14259c640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1441006a0 0x7fd14406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd142d9d640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 0x7fd14419ed90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00eb10 tx=0x7fd13c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c004b20 con 0x7fd144102250 2024-04-25T00:28:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14419f930 con 0x7fd144102250 2024-04-25T00:28:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd13c004cc0 con 0x7fd144102250 2024-04-25T00:28:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c010690 con 0x7fd144102250 2024-04-25T00:28:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.367+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14419fe80 con 0x7fd144102250 2024-04-25T00:28:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.368+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144107a90 con 0x7fd144102250 2024-04-25T00:28:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.369+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd13c005020 con 0x7fd144102250 2024-04-25T00:28:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.370+0000 7fd13b7fe640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd1300a2b90 0x7fd1300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.370+0000 7fd14259c640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd1300a2b90 0x7fd1300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.370+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd13c0cf600 con 0x7fd144102250 2024-04-25T00:28:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.370+0000 7fd14259c640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd1300a2b90 0x7fd1300a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd134032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.374+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd13c09c6b0 con 0x7fd144102250 2024-04-25T00:28:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.590+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd144061c10 con 0x7fd1300a2b90 2024-04-25T00:28:29.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.615+0000 7fd13b7fe640 1 -- 172.21.15.31:0/4078607223 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd144061c10 con 0x7fd1300a2b90 2024-04-25T00:28:29.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd1300a2b90 msgr2=0x7fd1300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd1300a2b90 0x7fd1300a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd134032c40 comp rx=0 tx=0).stop 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 msgr2=0x7fd14419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 0x7fd14419ed90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00eb10 tx=0x7fd13c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 shutdown_connections 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd1300a2b90 0x7fd1300a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd144102250 0x7fd14419ed90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144101890 0x7fd14419c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 --2- 172.21.15.31:0/4078607223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1441006a0 0x7fd14406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 >> 172.21.15.31:0/4078607223 conn(0x7fd1440fbc30 msgr2=0x7fd144104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 shutdown_connections 2024-04-25T00:28:29.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.618+0000 7fd148d1b640 1 -- 172.21.15.31:0/4078607223 wait complete. 2024-04-25T00:28:29.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 -- 172.21.15.31:0/150461737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d8102250 msgr2=0x7f39d8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 --2- 172.21.15.31:0/150461737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d8102250 0x7f39d8108ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009ca0 tx=0x7f39d002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 -- 172.21.15.31:0/150461737 shutdown_connections 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 --2- 172.21.15.31:0/150461737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d8102250 0x7f39d8108ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 --2- 172.21.15.31:0/150461737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d8101890 0x7f39d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 --2- 172.21.15.31:0/150461737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d81006a0 0x7f39d8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 -- 172.21.15.31:0/150461737 >> 172.21.15.31:0/150461737 conn(0x7f39d80fbc10 msgr2=0x7f39d80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 -- 172.21.15.31:0/150461737 shutdown_connections 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.735+0000 7f39df4b3640 1 -- 172.21.15.31:0/150461737 wait complete. 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 Processor -- start 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 -- start start 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 0x7f39d8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d8101890 0x7f39d8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d8102250 0x7f39d819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f39d810d2f0 con 0x7f39d81006a0 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39d810d170 con 0x7f39d8102250 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39df4b3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39d810d470 con 0x7f39d8101890 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39dd24f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 0x7f39d8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39dd24f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 0x7f39d8197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50212/0 (socket says 172.21.15.31:50212) 2024-04-25T00:28:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39dd24f640 1 -- 172.21.15.31:0/4288725399 learned_addr learned my addr 172.21.15.31:0/4288725399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39dca4e640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d8101890 0x7f39d8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.736+0000 7f39dda50640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d8102250 0x7f39d819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dd24f640 1 -- 172.21.15.31:0/4288725399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d8101890 msgr2=0x7f39d8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dd24f640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d8101890 0x7f39d8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dd24f640 1 -- 172.21.15.31:0/4288725399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d8102250 msgr2=0x7f39d819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dd24f640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d8102250 0x7f39d819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dd24f640 1 -- 172.21.15.31:0/4288725399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d819f570 con 0x7f39d81006a0 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dda50640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d8102250 0x7f39d819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39dd24f640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 0x7f39d8197950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c970 tx=0x7f39d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4004220 con 0x7f39d81006a0 2024-04-25T00:28:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d819f860 con 0x7f39d81006a0 2024-04-25T00:28:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39d40043c0 con 0x7f39d81006a0 2024-04-25T00:28:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400f450 con 0x7f39d81006a0 2024-04-25T00:28:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.737+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d819fdb0 con 0x7f39d81006a0 2024-04-25T00:28:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.738+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d8107a40 con 0x7f39d81006a0 2024-04-25T00:28:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.739+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39d40079b0 con 0x7f39d81006a0 2024-04-25T00:28:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.740+0000 7f39ce7fc640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f39c40a2b90 0x7f39c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.740+0000 7f39dca4e640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f39c40a2b90 0x7f39c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.740+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f39d40cecb0 con 0x7f39d81006a0 2024-04-25T00:28:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.740+0000 7f39dca4e640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f39c40a2b90 0x7f39c40a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39d8198e60 tx=0x7f39c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.745+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39d4014030 con 0x7f39d81006a0 2024-04-25T00:28:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.959+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f39d8061b80 con 0x7f39c40a2b90 2024-04-25T00:28:29.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.964+0000 7f39ce7fc640 1 -- 172.21.15.31:0/4288725399 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f39d8061b80 con 0x7f39c40a2b90 2024-04-25T00:28:29.965 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/89d93de1-8e6a-40dc-bf6e-3aa8107e8cdc 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f39c40a2b90 msgr2=0x7f39c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f39c40a2b90 0x7f39c40a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39d8198e60 tx=0x7f39c8005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 msgr2=0x7f39d8197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 0x7f39d8197950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c970 tx=0x7f39d400ce30 comp rx=0 tx=0).stop 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 shutdown_connections 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d8102250 0x7f39d819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d8101890 0x7f39d8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f39c40a2b90 0x7f39c40a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 --2- 172.21.15.31:0/4288725399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39d81006a0 0x7f39d8197950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:29.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 >> 172.21.15.31:0/4288725399 conn(0x7f39d80fbc10 msgr2=0x7f39d8104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:29.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 shutdown_connections 2024-04-25T00:28:29.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:29.966+0000 7f39df4b3640 1 -- 172.21.15.31:0/4288725399 wait complete. 2024-04-25T00:28:29.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 -- 172.21.15.31:0/2762420945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1029a0 msgr2=0x7f677c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 --2- 172.21.15.31:0/2762420945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1029a0 0x7f677c102e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6770009900 tx=0x7f677002a410 comp rx=0 tx=0).stop 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 -- 172.21.15.31:0/2762420945 shutdown_connections 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 --2- 172.21.15.31:0/2762420945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677c103360 0x7f677c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 --2- 172.21.15.31:0/2762420945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1029a0 0x7f677c102e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 --2- 172.21.15.31:0/2762420945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677c1017b0 0x7f677c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 -- 172.21.15.31:0/2762420945 >> 172.21.15.31:0/2762420945 conn(0x7f677c0fcd40 msgr2=0x7f677c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 -- 172.21.15.31:0/2762420945 shutdown_connections 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 -- 172.21.15.31:0/2762420945 wait complete. 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 Processor -- start 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.079+0000 7f67814cd640 1 -- start start 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 0x7f677c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677c1029a0 0x7f677c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677c103360 0x7f677c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d2c0 con 0x7f677c1017b0 2024-04-25T00:28:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d140 con 0x7f677c1029a0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d440 con 0x7f677c103360 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 0x7f677c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 0x7f677c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50240/0 (socket says 172.21.15.31:50240) 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 -- 172.21.15.31:0/4071190719 learned_addr learned my addr 172.21.15.31:0/4071190719 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677b7fe640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677c103360 0x7f677c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 -- 172.21.15.31:0/4071190719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677c103360 msgr2=0x7f677c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677c103360 0x7f677c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677a7fc640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677c1029a0 0x7f677c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 -- 172.21.15.31:0/4071190719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677c1029a0 msgr2=0x7f677c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677c1029a0 0x7f677c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 -- 172.21.15.31:0/4071190719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c19f620 con 0x7f677c1017b0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f677affd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 0x7f677c197a50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9c0 tx=0x7f676c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00cde0 con 0x7f677c1017b0 2024-04-25T00:28:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c19f910 con 0x7f677c1017b0 2024-04-25T00:28:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f676c0108e0 con 0x7f677c1017b0 2024-04-25T00:28:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0188e0 con 0x7f677c1017b0 2024-04-25T00:28:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.080+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c19fe60 con 0x7f677c1017b0 2024-04-25T00:28:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.081+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c108c40 con 0x7f677c1017b0 2024-04-25T00:28:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.083+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f676c01f020 con 0x7f677c1017b0 2024-04-25T00:28:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.083+0000 7f6763fff640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67640a2b90 0x7f67640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.083+0000 7f677a7fc640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67640a2b90 0x7f67640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.083+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f676c0cf780 con 0x7f677c1017b0 2024-04-25T00:28:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.084+0000 7f677a7fc640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67640a2b90 0x7f67640a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f677002a930 tx=0x7f6770000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.087+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f676c09c830 con 0x7f677c1017b0 2024-04-25T00:28:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.180+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f676c0987a0 con 0x7f677c1017b0 2024-04-25T00:28:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] up:active 2024-04-25T00:28:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T00:28:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:28:30.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: Cluster is now healthy 2024-04-25T00:28:30.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=j=up:active} 3 up:standby-replay 5 up:standby 2024-04-25T00:28:30.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-25T00:28:30.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:30 smithi031 ceph-mon[29167]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T00:28:30.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.319+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f677c02d170 con 0x7f677c1017b0 2024-04-25T00:28:30.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.319+0000 7f6763fff640 1 -- 172.21.15.31:0/4071190719 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11926 (secure 0 0 0) 0x7f676c0c3c60 con 0x7f677c1017b0 2024-04-25T00:28:30.320 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:30.320 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:28:30.181657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24512,24517,24499,24434,24424]},"id":1}]} 2024-04-25T00:28:30.320 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:28:30.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.321+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67640a2b90 msgr2=0x7f67640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.321+0000 7f67814cd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67640a2b90 0x7f67640a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f677002a930 tx=0x7f6770000bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.321+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 msgr2=0x7f677c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.321+0000 7f67814cd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 0x7f677c197a50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9c0 tx=0x7f676c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 shutdown_connections 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677c103360 0x7f677c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677c1029a0 0x7f677c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f67640a2b90 0x7f67640a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 --2- 172.21.15.31:0/4071190719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f677c1017b0 0x7f677c197a50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 >> 172.21.15.31:0/4071190719 conn(0x7f677c0fcd40 msgr2=0x7f677c105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 shutdown_connections 2024-04-25T00:28:30.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.322+0000 7f67814cd640 1 -- 172.21.15.31:0/4071190719 wait complete. 2024-04-25T00:28:30.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:28:30.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:30.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] up:active 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: Cluster is now healthy 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=j=up:active} 3 up:standby-replay 5 up:standby 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-mon[33112]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: 2024-04-25T00:28:30.051+0000 7f073c611700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: 2024-04-25T00:28:30.051+0000 7f073c611700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: 2024-04-25T00:28:30.051+0000 7f073c611700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T00:28:30.390 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:28:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: 2024-04-25T00:28:30.051+0000 7f073c611700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T00:28:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 -- 172.21.15.31:0/1955676624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 msgr2=0x7f23ac108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 --2- 172.21.15.31:0/1955676624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac108ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009ca0 tx=0x7f23a402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 -- 172.21.15.31:0/1955676624 shutdown_connections 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 --2- 172.21.15.31:0/1955676624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac108ae0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 --2- 172.21.15.31:0/1955676624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac101890 0x7f23ac101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 --2- 172.21.15.31:0/1955676624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac1006a0 0x7f23ac100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 -- 172.21.15.31:0/1955676624 >> 172.21.15.31:0/1955676624 conn(0x7f23ac0fbc30 msgr2=0x7f23ac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 -- 172.21.15.31:0/1955676624 shutdown_connections 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.435+0000 7f23b135a640 1 -- 172.21.15.31:0/1955676624 wait complete. 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 Processor -- start 2024-04-25T00:28:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 -- start start 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac1006a0 0x7f23ac197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac101890 0x7f23ac197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10c1d0 con 0x7f23ac102250 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10c050 con 0x7f23ac1006a0 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23b135a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10c350 con 0x7f23ac101890 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50250/0 (socket says 172.21.15.31:50250) 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23ab7fe640 1 -- 172.21.15.31:0/162358812 learned_addr learned my addr 172.21.15.31:0/162358812 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.436+0000 7f23aaffd640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac1006a0 0x7f23ac197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23aa7fc640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac101890 0x7f23ac197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23ab7fe640 1 -- 172.21.15.31:0/162358812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac101890 msgr2=0x7f23ac197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23ab7fe640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac101890 0x7f23ac197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23ab7fe640 1 -- 172.21.15.31:0/162358812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac1006a0 msgr2=0x7f23ac197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23ab7fe640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac1006a0 0x7f23ac197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23ab7fe640 1 -- 172.21.15.31:0/162358812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac19f580 con 0x7f23ac102250 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23aaffd640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac1006a0 0x7f23ac197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23ab7fe640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac19ef10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23a4031970 tx=0x7f23a4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4003d90 con 0x7f23ac102250 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac19f810 con 0x7f23ac102250 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a40044f0 con 0x7f23ac102250 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4030bd0 con 0x7f23ac102250 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.437+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac19fd60 con 0x7f23ac102250 2024-04-25T00:28:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.438+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac19ffc0 con 0x7f23ac102250 2024-04-25T00:28:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.439+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23a4004040 con 0x7f23ac102250 2024-04-25T00:28:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.440+0000 7f2393fff640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23940a2c20 0x7f23940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.440+0000 7f23aaffd640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23940a2c20 0x7f23940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.440+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f23a40f03d0 con 0x7f23ac102250 2024-04-25T00:28:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.440+0000 7f23aaffd640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23940a2c20 0x7f23940a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f23ac1016f0 tx=0x7f239c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:30.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.445+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f23a40bd4b0 con 0x7f23ac102250 2024-04-25T00:28:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.635+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23ac1a0350 con 0x7f23ac102250 2024-04-25T00:28:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.636+0000 7f2393fff640 1 -- 172.21.15.31:0/162358812 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22209 (secure 0 0 0) 0x7f23ac1a0350 con 0x7f23ac102250 2024-04-25T00:28:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:30.637 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:28:25.157023+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.638+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23940a2c20 msgr2=0x7f23940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.638+0000 7f23b135a640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23940a2c20 0x7f23940a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f23ac1016f0 tx=0x7f239c005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.638+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 msgr2=0x7f23ac19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.638+0000 7f23b135a640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac19ef10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23a4031970 tx=0x7f23a4003ba0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.638+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 shutdown_connections 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23940a2c20 0x7f23940a50e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23ac102250 0x7f23ac19ef10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac101890 0x7f23ac197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 --2- 172.21.15.31:0/162358812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23ac1006a0 0x7f23ac197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 >> 172.21.15.31:0/162358812 conn(0x7f23ac0fbc30 msgr2=0x7f23ac069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 shutdown_connections 2024-04-25T00:28:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.639+0000 7f23b135a640 1 -- 172.21.15.31:0/162358812 wait complete. 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] up:active 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: Cluster is now healthy 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=j=up:active} 3 up:standby-replay 5 up:standby 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-25T00:28:30.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:30 smithi040 ceph-mon[32945]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T00:28:30.651 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:28:30.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:28:30.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 -- 172.21.15.31:0/3294160978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 msgr2=0x7fa2c0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/3294160978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c0108ac0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009ca0 tx=0x7fa2b802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 -- 172.21.15.31:0/3294160978 shutdown_connections 2024-04-25T00:28:30.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/3294160978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c0108ac0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/3294160978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c0101870 0x7fa2c0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/3294160978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 0x7fa2c0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 -- 172.21.15.31:0/3294160978 >> 172.21.15.31:0/3294160978 conn(0x7fa2c00fbbf0 msgr2=0x7fa2c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 -- 172.21.15.31:0/3294160978 shutdown_connections 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.752+0000 7fa2c57d5640 1 -- 172.21.15.31:0/3294160978 wait complete. 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 Processor -- start 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 -- start start 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 0x7fa2c0069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c0101870 0x7fa2c019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c010c190 con 0x7fa2c0102230 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c010c010 con 0x7fa2c0101870 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2c57d5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c010c310 con 0x7fa2c0100680 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50268/0 (socket says 172.21.15.31:50268) 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 -- 172.21.15.31:0/1883729277 learned_addr learned my addr 172.21.15.31:0/1883729277 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2beffd640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 0x7fa2c0069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2b7fff640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c0101870 0x7fa2c019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 -- 172.21.15.31:0/1883729277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 msgr2=0x7fa2c0069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 0x7fa2c0069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 -- 172.21.15.31:0/1883729277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c0101870 msgr2=0x7fa2c019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c0101870 0x7fa2c019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2bf7fe640 1 -- 172.21.15.31:0/1883729277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c019f5e0 con 0x7fa2c0102230 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.753+0000 7fa2beffd640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 0x7fa2c0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.754+0000 7fa2bf7fe640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c019ecd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8031970 tx=0x7fa2b8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.754+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004040 con 0x7fa2c0102230 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.754+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b8004690 con 0x7fa2c0102230 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.754+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c019f810 con 0x7fa2c0102230 2024-04-25T00:28:30.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.754+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8030c60 con 0x7fa2c0102230 2024-04-25T00:28:30.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.754+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c019fdb0 con 0x7fa2c0102230 2024-04-25T00:28:30.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.755+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c01079f0 con 0x7fa2c0102230 2024-04-25T00:28:30.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.756+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa2b80041e0 con 0x7fa2c0102230 2024-04-25T00:28:30.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.757+0000 7fa2bcff9640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2ac0a2ca0 0x7fa2ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:30.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.757+0000 7fa2beffd640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2ac0a2ca0 0x7fa2ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:30.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.757+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa2b80f0310 con 0x7fa2c0102230 2024-04-25T00:28:30.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.757+0000 7fa2beffd640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2ac0a2ca0 0x7fa2ac0a5160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa2c01016d0 tx=0x7fa2b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:30.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.762+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa2b80bd3d0 con 0x7fa2c0102230 2024-04-25T00:28:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.947+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2c0061b80 con 0x7fa2ac0a2ca0 2024-04-25T00:28:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.954+0000 7fa2bcff9640 1 -- 172.21.15.31:0/1883729277 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258930 (secure 0 0 0) 0x7fa2c0061b80 con 0x7fa2ac0a2ca0 2024-04-25T00:28:30.959 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2ac0a2ca0 msgr2=0x7fa2ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2ac0a2ca0 0x7fa2ac0a5160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa2c01016d0 tx=0x7fa2b0005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 msgr2=0x7fa2c019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c019ecd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8031970 tx=0x7fa2b8003ba0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 shutdown_connections 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2ac0a2ca0 0x7fa2ac0a5160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2c0102230 0x7fa2c019ecd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c0101870 0x7fa2c019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 --2- 172.21.15.31:0/1883729277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c0100680 0x7fa2c0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:30.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.957+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 >> 172.21.15.31:0/1883729277 conn(0x7fa2c00fbbf0 msgr2=0x7fa2c0104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:30.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.958+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 shutdown_connections 2024-04-25T00:28:30.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:30.958+0000 7fa2c57d5640 1 -- 172.21.15.31:0/1883729277 wait complete. 2024-04-25T00:28:30.969 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-04-25T00:28:29.302384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":127,"num_read_kb":110,"num_write":279,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":209,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360392,"kb_used_data":11196,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581432,"statfs":{"total":1151940427776,"available":1151571386368,"internally_reserved":0,"allocated":11464704,"data_stored":7118086,"data_compressed":107184,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4640,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":98,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001225"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392588+0000","last_change":"2024-04-25T00:28:24.187326+0000","last_active":"2024-04-25T00:28:25.392588+0000","last_peered":"2024-04-25T00:28:25.392588+0000","last_clean":"2024-04-25T00:28:25.392588+0000","last_became_active":"2024-04-25T00:28:24.186536+0000","last_became_peered":"2024-04-25T00:28:24.186536+0000","last_unstale":"2024-04-25T00:28:25.392588+0000","last_undegraded":"2024-04-25T00:28:25.392588+0000","last_fullsized":"2024-04-25T00:28:25.392588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:25.035523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160508+0000","last_change":"2024-04-25T00:28:24.181763+0000","last_active":"2024-04-25T00:28:25.160508+0000","last_peered":"2024-04-25T00:28:25.160508+0000","last_clean":"2024-04-25T00:28:25.160508+0000","last_became_active":"2024-04-25T00:28:24.181437+0000","last_became_peered":"2024-04-25T00:28:24.181437+0000","last_unstale":"2024-04-25T00:28:25.160508+0000","last_undegraded":"2024-04-25T00:28:25.160508+0000","last_fullsized":"2024-04-25T00:28:25.160508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:54.019249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160575+0000","last_change":"2024-04-25T00:28:24.186648+0000","last_active":"2024-04-25T00:28:25.160575+0000","last_peered":"2024-04-25T00:28:25.160575+0000","last_clean":"2024-04-25T00:28:25.160575+0000","last_became_active":"2024-04-25T00:28:24.186303+0000","last_became_peered":"2024-04-25T00:28:24.186303+0000","last_unstale":"2024-04-25T00:28:25.160575+0000","last_undegraded":"2024-04-25T00:28:25.160575+0000","last_fullsized":"2024-04-25T00:28:25.160575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:14.521970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161448+0000","last_change":"2024-04-25T00:28:24.179138+0000","last_active":"2024-04-25T00:28:25.161448+0000","last_peered":"2024-04-25T00:28:25.161448+0000","last_clean":"2024-04-25T00:28:25.161448+0000","last_became_active":"2024-04-25T00:28:24.178664+0000","last_became_peered":"2024-04-25T00:28:24.178664+0000","last_unstale":"2024-04-25T00:28:25.161448+0000","last_undegraded":"2024-04-25T00:28:25.161448+0000","last_fullsized":"2024-04-25T00:28:25.161448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:07.869662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161400+0000","last_change":"2024-04-25T00:28:24.174081+0000","last_active":"2024-04-25T00:28:25.161400+0000","last_peered":"2024-04-25T00:28:25.161400+0000","last_clean":"2024-04-25T00:28:25.161400+0000","last_became_active":"2024-04-25T00:28:24.173670+0000","last_became_peered":"2024-04-25T00:28:24.173670+0000","last_unstale":"2024-04-25T00:28:25.161400+0000","last_undegraded":"2024-04-25T00:28:25.161400+0000","last_fullsized":"2024-04-25T00:28:25.161400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:43.255598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355757+0000","last_change":"2024-04-25T00:28:24.174664+0000","last_active":"2024-04-25T00:28:26.355757+0000","last_peered":"2024-04-25T00:28:26.355757+0000","last_clean":"2024-04-25T00:28:26.355757+0000","last_became_active":"2024-04-25T00:28:24.174238+0000","last_became_peered":"2024-04-25T00:28:24.174238+0000","last_unstale":"2024-04-25T00:28:26.355757+0000","last_undegraded":"2024-04-25T00:28:26.355757+0000","last_fullsized":"2024-04-25T00:28:26.355757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:21.214976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.186706+0000","last_change":"2024-04-25T00:28:24.181123+0000","last_active":"2024-04-25T00:28:24.186706+0000","last_peered":"2024-04-25T00:28:24.186706+0000","last_clean":"2024-04-25T00:28:24.186706+0000","last_became_active":"2024-04-25T00:28:24.179516+0000","last_became_peered":"2024-04-25T00:28:24.179516+0000","last_unstale":"2024-04-25T00:28:24.186706+0000","last_undegraded":"2024-04-25T00:28:24.186706+0000","last_fullsized":"2024-04-25T00:28:24.186706+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:33.635846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.413755+0000","last_change":"2024-04-25T00:28:24.179964+0000","last_active":"2024-04-25T00:28:25.413755+0000","last_peered":"2024-04-25T00:28:25.413755+0000","last_clean":"2024-04-25T00:28:25.413755+0000","last_became_active":"2024-04-25T00:28:24.179585+0000","last_became_peered":"2024-04-25T00:28:24.179585+0000","last_unstale":"2024-04-25T00:28:25.413755+0000","last_undegraded":"2024-04-25T00:28:25.413755+0000","last_fullsized":"2024-04-25T00:28:25.413755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:42.990145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.356167+0000","last_change":"2024-04-25T00:28:24.170818+0000","last_active":"2024-04-25T00:28:26.356167+0000","last_peered":"2024-04-25T00:28:26.356167+0000","last_clean":"2024-04-25T00:28:26.356167+0000","last_became_active":"2024-04-25T00:28:24.170412+0000","last_became_peered":"2024-04-25T00:28:24.170412+0000","last_unstale":"2024-04-25T00:28:26.356167+0000","last_undegraded":"2024-04-25T00:28:26.356167+0000","last_fullsized":"2024-04-25T00:28:26.356167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:20.456317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355533+0000","last_change":"2024-04-25T00:28:24.182509+0000","last_active":"2024-04-25T00:28:26.355533+0000","last_peered":"2024-04-25T00:28:26.355533+0000","last_clean":"2024-04-25T00:28:26.355533+0000","last_became_active":"2024-04-25T00:28:24.182141+0000","last_became_peered":"2024-04-25T00:28:24.182141+0000","last_unstale":"2024-04-25T00:28:26.355533+0000","last_undegraded":"2024-04-25T00:28:26.355533+0000","last_fullsized":"2024-04-25T00:28:26.355533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:33.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.190496+0000","last_change":"2024-04-25T00:28:24.189265+0000","last_active":"2024-04-25T00:28:24.190496+0000","last_peered":"2024-04-25T00:28:24.190496+0000","last_clean":"2024-04-25T00:28:24.190496+0000","last_became_active":"2024-04-25T00:28:24.188829+0000","last_became_peered":"2024-04-25T00:28:24.188829+0000","last_unstale":"2024-04-25T00:28:24.190496+0000","last_undegraded":"2024-04-25T00:28:24.190496+0000","last_fullsized":"2024-04-25T00:28:24.190496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:13.125851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161340+0000","last_change":"2024-04-25T00:28:24.180145+0000","last_active":"2024-04-25T00:28:25.161340+0000","last_peered":"2024-04-25T00:28:25.161340+0000","last_clean":"2024-04-25T00:28:25.161340+0000","last_became_active":"2024-04-25T00:28:24.179802+0000","last_became_peered":"2024-04-25T00:28:24.179802+0000","last_unstale":"2024-04-25T00:28:25.161340+0000","last_undegraded":"2024-04-25T00:28:25.161340+0000","last_fullsized":"2024-04-25T00:28:25.161340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:34.367575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354937+0000","last_change":"2024-04-25T00:28:24.183769+0000","last_active":"2024-04-25T00:28:26.354937+0000","last_peered":"2024-04-25T00:28:26.354937+0000","last_clean":"2024-04-25T00:28:26.354937+0000","last_became_active":"2024-04-25T00:28:24.183432+0000","last_became_peered":"2024-04-25T00:28:24.183432+0000","last_unstale":"2024-04-25T00:28:26.354937+0000","last_undegraded":"2024-04-25T00:28:26.354937+0000","last_fullsized":"2024-04-25T00:28:26.354937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:27.656794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160247+0000","last_change":"2024-04-25T00:28:24.184364+0000","last_active":"2024-04-25T00:28:25.160247+0000","last_peered":"2024-04-25T00:28:25.160247+0000","last_clean":"2024-04-25T00:28:25.160247+0000","last_became_active":"2024-04-25T00:28:24.184038+0000","last_became_peered":"2024-04-25T00:28:24.184038+0000","last_unstale":"2024-04-25T00:28:25.160247+0000","last_undegraded":"2024-04-25T00:28:25.160247+0000","last_fullsized":"2024-04-25T00:28:25.160247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:50.495466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160091+0000","last_change":"2024-04-25T00:28:24.180236+0000","last_active":"2024-04-25T00:28:25.160091+0000","last_peered":"2024-04-25T00:28:25.160091+0000","last_clean":"2024-04-25T00:28:25.160091+0000","last_became_active":"2024-04-25T00:28:24.179929+0000","last_became_peered":"2024-04-25T00:28:24.179929+0000","last_unstale":"2024-04-25T00:28:25.160091+0000","last_undegraded":"2024-04-25T00:28:25.160091+0000","last_fullsized":"2024-04-25T00:28:25.160091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:29.858564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354243+0000","last_change":"2024-04-25T00:28:24.185605+0000","last_active":"2024-04-25T00:28:26.354243+0000","last_peered":"2024-04-25T00:28:26.354243+0000","last_clean":"2024-04-25T00:28:26.354243+0000","last_became_active":"2024-04-25T00:28:24.185270+0000","last_became_peered":"2024-04-25T00:28:24.185270+0000","last_unstale":"2024-04-25T00:28:26.354243+0000","last_undegraded":"2024-04-25T00:28:26.354243+0000","last_fullsized":"2024-04-25T00:28:26.354243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:58.798429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392710+0000","last_change":"2024-04-25T00:28:24.187625+0000","last_active":"2024-04-25T00:28:25.392710+0000","last_peered":"2024-04-25T00:28:25.392710+0000","last_clean":"2024-04-25T00:28:25.392710+0000","last_became_active":"2024-04-25T00:28:24.186961+0000","last_became_peered":"2024-04-25T00:28:24.186961+0000","last_unstale":"2024-04-25T00:28:25.392710+0000","last_undegraded":"2024-04-25T00:28:25.392710+0000","last_fullsized":"2024-04-25T00:28:25.392710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:26.099099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411375+0000","last_change":"2024-04-25T00:28:24.177342+0000","last_active":"2024-04-25T00:28:25.411375+0000","last_peered":"2024-04-25T00:28:25.411375+0000","last_clean":"2024-04-25T00:28:25.411375+0000","last_became_active":"2024-04-25T00:28:24.176763+0000","last_became_peered":"2024-04-25T00:28:24.176763+0000","last_unstale":"2024-04-25T00:28:25.411375+0000","last_undegraded":"2024-04-25T00:28:25.411375+0000","last_fullsized":"2024-04-25T00:28:25.411375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:47.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.187348+0000","last_change":"2024-04-25T00:28:24.186069+0000","last_active":"2024-04-25T00:28:24.187348+0000","last_peered":"2024-04-25T00:28:24.187348+0000","last_clean":"2024-04-25T00:28:24.187348+0000","last_became_active":"2024-04-25T00:28:24.185713+0000","last_became_peered":"2024-04-25T00:28:24.185713+0000","last_unstale":"2024-04-25T00:28:24.187348+0000","last_undegraded":"2024-04-25T00:28:24.187348+0000","last_fullsized":"2024-04-25T00:28:24.187348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:21.862359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411258+0000","last_change":"2024-04-25T00:28:23.402187+0000","last_active":"2024-04-25T00:28:25.411258+0000","last_peered":"2024-04-25T00:28:25.411258+0000","last_clean":"2024-04-25T00:28:25.411258+0000","last_became_active":"2024-04-25T00:28:23.401868+0000","last_became_peered":"2024-04-25T00:28:23.401868+0000","last_unstale":"2024-04-25T00:28:25.411258+0000","last_undegraded":"2024-04-25T00:28:25.411258+0000","last_fullsized":"2024-04-25T00:28:25.411258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:44.022628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414232+0000","last_change":"2024-04-25T00:28:24.180731+0000","last_active":"2024-04-25T00:28:25.414232+0000","last_peered":"2024-04-25T00:28:25.414232+0000","last_clean":"2024-04-25T00:28:25.414232+0000","last_became_active":"2024-04-25T00:28:24.180274+0000","last_became_peered":"2024-04-25T00:28:24.180274+0000","last_unstale":"2024-04-25T00:28:25.414232+0000","last_undegraded":"2024-04-25T00:28:25.414232+0000","last_fullsized":"2024-04-25T00:28:25.414232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:46.102326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414196+0000","last_change":"2024-04-25T00:28:23.405683+0000","last_active":"2024-04-25T00:28:25.414196+0000","last_peered":"2024-04-25T00:28:25.414196+0000","last_clean":"2024-04-25T00:28:25.414196+0000","last_became_active":"2024-04-25T00:28:23.405305+0000","last_became_peered":"2024-04-25T00:28:23.405305+0000","last_unstale":"2024-04-25T00:28:25.414196+0000","last_undegraded":"2024-04-25T00:28:25.414196+0000","last_fullsized":"2024-04-25T00:28:25.414196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:39.782047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160676+0000","last_change":"2024-04-25T00:28:24.187565+0000","last_active":"2024-04-25T00:28:25.160676+0000","last_peered":"2024-04-25T00:28:25.160676+0000","last_clean":"2024-04-25T00:28:25.160676+0000","last_became_active":"2024-04-25T00:28:24.186397+0000","last_became_peered":"2024-04-25T00:28:24.186397+0000","last_unstale":"2024-04-25T00:28:25.160676+0000","last_undegraded":"2024-04-25T00:28:25.160676+0000","last_fullsized":"2024-04-25T00:28:25.160676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:36.750031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.442562+0000","last_change":"2024-04-25T00:28:23.398726+0000","last_active":"2024-04-25T00:28:27.442562+0000","last_peered":"2024-04-25T00:28:27.442562+0000","last_clean":"2024-04-25T00:28:27.442562+0000","last_became_active":"2024-04-25T00:28:23.398370+0000","last_became_peered":"2024-04-25T00:28:23.398370+0000","last_unstale":"2024-04-25T00:28:27.442562+0000","last_undegraded":"2024-04-25T00:28:27.442562+0000","last_fullsized":"2024-04-25T00:28:27.442562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:19.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354678+0000","last_change":"2024-04-25T00:28:24.181424+0000","last_active":"2024-04-25T00:28:26.354678+0000","last_peered":"2024-04-25T00:28:26.354678+0000","last_clean":"2024-04-25T00:28:26.354678+0000","last_became_active":"2024-04-25T00:28:24.179077+0000","last_became_peered":"2024-04-25T00:28:24.179077+0000","last_unstale":"2024-04-25T00:28:26.354678+0000","last_undegraded":"2024-04-25T00:28:26.354678+0000","last_fullsized":"2024-04-25T00:28:26.354678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.926451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161493+0000","last_change":"2024-04-25T00:28:23.402419+0000","last_active":"2024-04-25T00:28:25.161493+0000","last_peered":"2024-04-25T00:28:25.161493+0000","last_clean":"2024-04-25T00:28:25.161493+0000","last_became_active":"2024-04-25T00:28:23.402070+0000","last_became_peered":"2024-04-25T00:28:23.402070+0000","last_unstale":"2024-04-25T00:28:25.161493+0000","last_undegraded":"2024-04-25T00:28:25.161493+0000","last_fullsized":"2024-04-25T00:28:25.161493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:08.799577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354865+0000","last_change":"2024-04-25T00:28:24.178856+0000","last_active":"2024-04-25T00:28:26.354865+0000","last_peered":"2024-04-25T00:28:26.354865+0000","last_clean":"2024-04-25T00:28:26.354865+0000","last_became_active":"2024-04-25T00:28:24.178469+0000","last_became_peered":"2024-04-25T00:28:24.178469+0000","last_unstale":"2024-04-25T00:28:26.354865+0000","last_undegraded":"2024-04-25T00:28:26.354865+0000","last_fullsized":"2024-04-25T00:28:26.354865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:28.211603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.436956+0000","last_change":"2024-04-25T00:28:23.400250+0000","last_active":"2024-04-25T00:28:26.436956+0000","last_peered":"2024-04-25T00:28:26.436956+0000","last_clean":"2024-04-25T00:28:26.436956+0000","last_became_active":"2024-04-25T00:28:23.399874+0000","last_became_peered":"2024-04-25T00:28:23.399874+0000","last_unstale":"2024-04-25T00:28:26.436956+0000","last_undegraded":"2024-04-25T00:28:26.436956+0000","last_fullsized":"2024-04-25T00:28:26.436956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:08.332441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161366+0000","last_change":"2024-04-25T00:28:24.170176+0000","last_active":"2024-04-25T00:28:25.161366+0000","last_peered":"2024-04-25T00:28:25.161366+0000","last_clean":"2024-04-25T00:28:25.161366+0000","last_became_active":"2024-04-25T00:28:24.169493+0000","last_became_peered":"2024-04-25T00:28:24.169493+0000","last_unstale":"2024-04-25T00:28:25.161366+0000","last_undegraded":"2024-04-25T00:28:25.161366+0000","last_fullsized":"2024-04-25T00:28:25.161366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:43.855777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.444321+0000","last_change":"2024-04-25T00:28:23.398003+0000","last_active":"2024-04-25T00:28:27.444321+0000","last_peered":"2024-04-25T00:28:27.444321+0000","last_clean":"2024-04-25T00:28:27.444321+0000","last_became_active":"2024-04-25T00:28:23.397229+0000","last_became_peered":"2024-04-25T00:28:23.397229+0000","last_unstale":"2024-04-25T00:28:27.444321+0000","last_undegraded":"2024-04-25T00:28:27.444321+0000","last_fullsized":"2024-04-25T00:28:27.444321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:59.475252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354752+0000","last_change":"2024-04-25T00:28:24.180899+0000","last_active":"2024-04-25T00:28:26.354752+0000","last_peered":"2024-04-25T00:28:26.354752+0000","last_clean":"2024-04-25T00:28:26.354752+0000","last_became_active":"2024-04-25T00:28:24.178987+0000","last_became_peered":"2024-04-25T00:28:24.178987+0000","last_unstale":"2024-04-25T00:28:26.354752+0000","last_undegraded":"2024-04-25T00:28:26.354752+0000","last_fullsized":"2024-04-25T00:28:26.354752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:22.561636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.443204+0000","last_change":"2024-04-25T00:28:23.397855+0000","last_active":"2024-04-25T00:28:27.443204+0000","last_peered":"2024-04-25T00:28:27.443204+0000","last_clean":"2024-04-25T00:28:27.443204+0000","last_became_active":"2024-04-25T00:28:23.397465+0000","last_became_peered":"2024-04-25T00:28:23.397465+0000","last_unstale":"2024-04-25T00:28:27.443204+0000","last_undegraded":"2024-04-25T00:28:27.443204+0000","last_fullsized":"2024-04-25T00:28:27.443204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.357397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.413711+0000","last_change":"2024-04-25T00:28:24.172890+0000","last_active":"2024-04-25T00:28:25.413711+0000","last_peered":"2024-04-25T00:28:25.413711+0000","last_clean":"2024-04-25T00:28:25.413711+0000","last_became_active":"2024-04-25T00:28:24.172250+0000","last_became_peered":"2024-04-25T00:28:24.172250+0000","last_unstale":"2024-04-25T00:28:25.413711+0000","last_undegraded":"2024-04-25T00:28:25.413711+0000","last_fullsized":"2024-04-25T00:28:25.413711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:34.372811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435327+0000","last_change":"2024-04-25T00:28:23.401140+0000","last_active":"2024-04-25T00:28:26.435327+0000","last_peered":"2024-04-25T00:28:26.435327+0000","last_clean":"2024-04-25T00:28:26.435327+0000","last_became_active":"2024-04-25T00:28:23.400804+0000","last_became_peered":"2024-04-25T00:28:23.400804+0000","last_unstale":"2024-04-25T00:28:26.435327+0000","last_undegraded":"2024-04-25T00:28:26.435327+0000","last_fullsized":"2024-04-25T00:28:26.435327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:00.044141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411401+0000","last_change":"2024-04-25T00:28:24.175596+0000","last_active":"2024-04-25T00:28:25.411401+0000","last_peered":"2024-04-25T00:28:25.411401+0000","last_clean":"2024-04-25T00:28:25.411401+0000","last_became_active":"2024-04-25T00:28:24.175085+0000","last_became_peered":"2024-04-25T00:28:24.175085+0000","last_unstale":"2024-04-25T00:28:25.411401+0000","last_undegraded":"2024-04-25T00:28:25.411401+0000","last_fullsized":"2024-04-25T00:28:25.411401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:05.801995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433879+0000","last_change":"2024-04-25T00:28:23.399427+0000","last_active":"2024-04-25T00:28:26.433879+0000","last_peered":"2024-04-25T00:28:26.433879+0000","last_clean":"2024-04-25T00:28:26.433879+0000","last_became_active":"2024-04-25T00:28:23.399067+0000","last_became_peered":"2024-04-25T00:28:23.399067+0000","last_unstale":"2024-04-25T00:28:26.433879+0000","last_undegraded":"2024-04-25T00:28:26.433879+0000","last_fullsized":"2024-04-25T00:28:26.433879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:06.853292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355993+0000","last_change":"2024-04-25T00:28:24.177367+0000","last_active":"2024-04-25T00:28:26.355993+0000","last_peered":"2024-04-25T00:28:26.355993+0000","last_clean":"2024-04-25T00:28:26.355993+0000","last_became_active":"2024-04-25T00:28:24.176890+0000","last_became_peered":"2024-04-25T00:28:24.176890+0000","last_unstale":"2024-04-25T00:28:26.355993+0000","last_undegraded":"2024-04-25T00:28:26.355993+0000","last_fullsized":"2024-04-25T00:28:26.355993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:27.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.167916+0000","last_change":"2024-04-25T00:28:23.409705+0000","last_active":"2024-04-25T00:28:24.167916+0000","last_peered":"2024-04-25T00:28:24.167916+0000","last_clean":"2024-04-25T00:28:24.167916+0000","last_became_active":"2024-04-25T00:28:23.408596+0000","last_became_peered":"2024-04-25T00:28:23.408596+0000","last_unstale":"2024-04-25T00:28:24.167916+0000","last_undegraded":"2024-04-25T00:28:24.167916+0000","last_fullsized":"2024-04-25T00:28:24.167916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:25.684497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160686+0000","last_change":"2024-04-25T00:28:24.179361+0000","last_active":"2024-04-25T00:28:25.160686+0000","last_peered":"2024-04-25T00:28:25.160686+0000","last_clean":"2024-04-25T00:28:25.160686+0000","last_became_active":"2024-04-25T00:28:24.179019+0000","last_became_peered":"2024-04-25T00:28:24.179019+0000","last_unstale":"2024-04-25T00:28:25.160686+0000","last_undegraded":"2024-04-25T00:28:25.160686+0000","last_fullsized":"2024-04-25T00:28:25.160686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:08.543283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433353+0000","last_change":"2024-04-25T00:28:23.404051+0000","last_active":"2024-04-25T00:28:26.433353+0000","last_peered":"2024-04-25T00:28:26.433353+0000","last_clean":"2024-04-25T00:28:26.433353+0000","last_became_active":"2024-04-25T00:28:23.403728+0000","last_became_peered":"2024-04-25T00:28:23.403728+0000","last_unstale":"2024-04-25T00:28:26.433353+0000","last_undegraded":"2024-04-25T00:28:26.433353+0000","last_fullsized":"2024-04-25T00:28:26.433353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:27.403464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355028+0000","last_change":"2024-04-25T00:28:24.178506+0000","last_active":"2024-04-25T00:28:26.355028+0000","last_peered":"2024-04-25T00:28:26.355028+0000","last_clean":"2024-04-25T00:28:26.355028+0000","last_became_active":"2024-04-25T00:28:24.177053+0000","last_became_peered":"2024-04-25T00:28:24.177053+0000","last_unstale":"2024-04-25T00:28:26.355028+0000","last_undegraded":"2024-04-25T00:28:26.355028+0000","last_fullsized":"2024-04-25T00:28:26.355028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:04.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433978+0000","last_change":"2024-04-25T00:28:23.400563+0000","last_active":"2024-04-25T00:28:26.433978+0000","last_peered":"2024-04-25T00:28:26.433978+0000","last_clean":"2024-04-25T00:28:26.433978+0000","last_became_active":"2024-04-25T00:28:23.399982+0000","last_became_peered":"2024-04-25T00:28:23.399982+0000","last_unstale":"2024-04-25T00:28:26.433978+0000","last_undegraded":"2024-04-25T00:28:26.433978+0000","last_fullsized":"2024-04-25T00:28:26.433978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:01.652027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161588+0000","last_change":"2024-04-25T00:28:24.179687+0000","last_active":"2024-04-25T00:28:25.161588+0000","last_peered":"2024-04-25T00:28:25.161588+0000","last_clean":"2024-04-25T00:28:25.161588+0000","last_became_active":"2024-04-25T00:28:24.179345+0000","last_became_peered":"2024-04-25T00:28:24.179345+0000","last_unstale":"2024-04-25T00:28:25.161588+0000","last_undegraded":"2024-04-25T00:28:25.161588+0000","last_fullsized":"2024-04-25T00:28:25.161588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:49.807417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.417344+0000","last_change":"2024-04-25T00:28:23.401353+0000","last_active":"2024-04-25T00:28:24.417344+0000","last_peered":"2024-04-25T00:28:24.417344+0000","last_clean":"2024-04-25T00:28:24.417344+0000","last_became_active":"2024-04-25T00:28:23.400985+0000","last_became_peered":"2024-04-25T00:28:23.400985+0000","last_unstale":"2024-04-25T00:28:24.417344+0000","last_undegraded":"2024-04-25T00:28:24.417344+0000","last_fullsized":"2024-04-25T00:28:24.417344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.277514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354636+0000","last_change":"2024-04-25T00:28:24.178796+0000","last_active":"2024-04-25T00:28:26.354636+0000","last_peered":"2024-04-25T00:28:26.354636+0000","last_clean":"2024-04-25T00:28:26.354636+0000","last_became_active":"2024-04-25T00:28:24.178457+0000","last_became_peered":"2024-04-25T00:28:24.178457+0000","last_unstale":"2024-04-25T00:28:26.354636+0000","last_undegraded":"2024-04-25T00:28:26.354636+0000","last_fullsized":"2024-04-25T00:28:26.354636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:48.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435499+0000","last_change":"2024-04-25T00:28:23.406046+0000","last_active":"2024-04-25T00:28:26.435499+0000","last_peered":"2024-04-25T00:28:26.435499+0000","last_clean":"2024-04-25T00:28:26.435499+0000","last_became_active":"2024-04-25T00:28:23.405686+0000","last_became_peered":"2024-04-25T00:28:23.405686+0000","last_unstale":"2024-04-25T00:28:26.435499+0000","last_undegraded":"2024-04-25T00:28:26.435499+0000","last_fullsized":"2024-04-25T00:28:26.435499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:30.551884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.189443+0000","last_change":"2024-04-25T00:28:24.188251+0000","last_active":"2024-04-25T00:28:24.189443+0000","last_peered":"2024-04-25T00:28:24.189443+0000","last_clean":"2024-04-25T00:28:24.189443+0000","last_became_active":"2024-04-25T00:28:24.187876+0000","last_became_peered":"2024-04-25T00:28:24.187876+0000","last_unstale":"2024-04-25T00:28:24.189443+0000","last_undegraded":"2024-04-25T00:28:24.189443+0000","last_fullsized":"2024-04-25T00:28:24.189443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:35.667382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.441291+0000","last_change":"2024-04-25T00:28:23.407240+0000","last_active":"2024-04-25T00:28:27.441291+0000","last_peered":"2024-04-25T00:28:27.441291+0000","last_clean":"2024-04-25T00:28:27.441291+0000","last_became_active":"2024-04-25T00:28:23.406794+0000","last_became_peered":"2024-04-25T00:28:23.406794+0000","last_unstale":"2024-04-25T00:28:27.441291+0000","last_undegraded":"2024-04-25T00:28:27.441291+0000","last_fullsized":"2024-04-25T00:28:27.441291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:35.861733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.187182+0000","last_change":"2024-04-25T00:28:24.183784+0000","last_active":"2024-04-25T00:28:24.187182+0000","last_peered":"2024-04-25T00:28:24.187182+0000","last_clean":"2024-04-25T00:28:24.187182+0000","last_became_active":"2024-04-25T00:28:24.183417+0000","last_became_peered":"2024-04-25T00:28:24.183417+0000","last_unstale":"2024-04-25T00:28:24.187182+0000","last_undegraded":"2024-04-25T00:28:24.187182+0000","last_fullsized":"2024-04-25T00:28:24.187182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:05.995155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.444731+0000","last_change":"2024-04-25T00:28:23.406324+0000","last_active":"2024-04-25T00:28:27.444731+0000","last_peered":"2024-04-25T00:28:27.444731+0000","last_clean":"2024-04-25T00:28:27.444731+0000","last_became_active":"2024-04-25T00:28:23.405964+0000","last_became_peered":"2024-04-25T00:28:23.405964+0000","last_unstale":"2024-04-25T00:28:27.444731+0000","last_undegraded":"2024-04-25T00:28:27.444731+0000","last_fullsized":"2024-04-25T00:28:27.444731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:17.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354614+0000","last_change":"2024-04-25T00:28:24.177898+0000","last_active":"2024-04-25T00:28:26.354614+0000","last_peered":"2024-04-25T00:28:26.354614+0000","last_clean":"2024-04-25T00:28:26.354614+0000","last_became_active":"2024-04-25T00:28:24.177578+0000","last_became_peered":"2024-04-25T00:28:24.177578+0000","last_unstale":"2024-04-25T00:28:26.354614+0000","last_undegraded":"2024-04-25T00:28:26.354614+0000","last_fullsized":"2024-04-25T00:28:26.354614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:36.294161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161599+0000","last_change":"2024-04-25T00:28:23.399570+0000","last_active":"2024-04-25T00:28:25.161599+0000","last_peered":"2024-04-25T00:28:25.161599+0000","last_clean":"2024-04-25T00:28:25.161599+0000","last_became_active":"2024-04-25T00:28:23.399230+0000","last_became_peered":"2024-04-25T00:28:23.399230+0000","last_unstale":"2024-04-25T00:28:25.161599+0000","last_undegraded":"2024-04-25T00:28:25.161599+0000","last_fullsized":"2024-04-25T00:28:25.161599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:46.850590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392493+0000","last_change":"2024-04-25T00:28:24.186113+0000","last_active":"2024-04-25T00:28:25.392493+0000","last_peered":"2024-04-25T00:28:25.392493+0000","last_clean":"2024-04-25T00:28:25.392493+0000","last_became_active":"2024-04-25T00:28:24.185743+0000","last_became_peered":"2024-04-25T00:28:24.185743+0000","last_unstale":"2024-04-25T00:28:25.392493+0000","last_undegraded":"2024-04-25T00:28:25.392493+0000","last_fullsized":"2024-04-25T00:28:25.392493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:10.973285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411322+0000","last_change":"2024-04-25T00:28:23.396054+0000","last_active":"2024-04-25T00:28:25.411322+0000","last_peered":"2024-04-25T00:28:25.411322+0000","last_clean":"2024-04-25T00:28:25.411322+0000","last_became_active":"2024-04-25T00:28:23.395707+0000","last_became_peered":"2024-04-25T00:28:23.395707+0000","last_unstale":"2024-04-25T00:28:25.411322+0000","last_undegraded":"2024-04-25T00:28:25.411322+0000","last_fullsized":"2024-04-25T00:28:25.411322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:20.169476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355165+0000","last_change":"2024-04-25T00:28:24.185655+0000","last_active":"2024-04-25T00:28:26.355165+0000","last_peered":"2024-04-25T00:28:26.355165+0000","last_clean":"2024-04-25T00:28:26.355165+0000","last_became_active":"2024-04-25T00:28:24.185123+0000","last_became_peered":"2024-04-25T00:28:24.185123+0000","last_unstale":"2024-04-25T00:28:26.355165+0000","last_undegraded":"2024-04-25T00:28:26.355165+0000","last_fullsized":"2024-04-25T00:28:26.355165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.845762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.171654+0000","last_change":"2024-04-25T00:28:23.410964+0000","last_active":"2024-04-25T00:28:24.171654+0000","last_peered":"2024-04-25T00:28:24.171654+0000","last_clean":"2024-04-25T00:28:24.171654+0000","last_became_active":"2024-04-25T00:28:23.410602+0000","last_became_peered":"2024-04-25T00:28:23.410602+0000","last_unstale":"2024-04-25T00:28:24.171654+0000","last_undegraded":"2024-04-25T00:28:24.171654+0000","last_fullsized":"2024-04-25T00:28:24.171654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:24.495790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355802+0000","last_change":"2024-04-25T00:28:24.184710+0000","last_active":"2024-04-25T00:28:26.355802+0000","last_peered":"2024-04-25T00:28:26.355802+0000","last_clean":"2024-04-25T00:28:26.355802+0000","last_became_active":"2024-04-25T00:28:24.184354+0000","last_became_peered":"2024-04-25T00:28:24.184354+0000","last_unstale":"2024-04-25T00:28:26.355802+0000","last_undegraded":"2024-04-25T00:28:26.355802+0000","last_fullsized":"2024-04-25T00:28:26.355802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:47.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354533+0000","last_change":"2024-04-25T00:28:24.180268+0000","last_active":"2024-04-25T00:28:26.354533+0000","last_peered":"2024-04-25T00:28:26.354533+0000","last_clean":"2024-04-25T00:28:26.354533+0000","last_became_active":"2024-04-25T00:28:24.178941+0000","last_became_peered":"2024-04-25T00:28:24.178941+0000","last_unstale":"2024-04-25T00:28:26.354533+0000","last_undegraded":"2024-04-25T00:28:26.354533+0000","last_fullsized":"2024-04-25T00:28:26.354533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:06.861257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.444421+0000","last_change":"2024-04-25T00:28:23.401437+0000","last_active":"2024-04-25T00:28:27.444421+0000","last_peered":"2024-04-25T00:28:27.444421+0000","last_clean":"2024-04-25T00:28:27.444421+0000","last_became_active":"2024-04-25T00:28:23.401091+0000","last_became_peered":"2024-04-25T00:28:23.401091+0000","last_unstale":"2024-04-25T00:28:27.444421+0000","last_undegraded":"2024-04-25T00:28:27.444421+0000","last_fullsized":"2024-04-25T00:28:27.444421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:58.748461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414209+0000","last_change":"2024-04-25T00:28:24.166224+0000","last_active":"2024-04-25T00:28:25.414209+0000","last_peered":"2024-04-25T00:28:25.414209+0000","last_clean":"2024-04-25T00:28:25.414209+0000","last_became_active":"2024-04-25T00:28:24.165626+0000","last_became_peered":"2024-04-25T00:28:24.165626+0000","last_unstale":"2024-04-25T00:28:25.414209+0000","last_undegraded":"2024-04-25T00:28:25.414209+0000","last_fullsized":"2024-04-25T00:28:25.414209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:04.956168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435448+0000","last_change":"2024-04-25T00:28:23.408551+0000","last_active":"2024-04-25T00:28:26.435448+0000","last_peered":"2024-04-25T00:28:26.435448+0000","last_clean":"2024-04-25T00:28:26.435448+0000","last_became_active":"2024-04-25T00:28:23.408137+0000","last_became_peered":"2024-04-25T00:28:23.408137+0000","last_unstale":"2024-04-25T00:28:26.435448+0000","last_undegraded":"2024-04-25T00:28:26.435448+0000","last_fullsized":"2024-04-25T00:28:26.435448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:53.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.174005+0000","last_change":"2024-04-25T00:27:10.587192+0000","last_active":"2024-04-25T00:28:24.174005+0000","last_peered":"2024-04-25T00:28:24.174005+0000","last_clean":"2024-04-25T00:28:24.174005+0000","last_became_active":"2024-04-25T00:27:10.586481+0000","last_became_peered":"2024-04-25T00:27:10.586481+0000","last_unstale":"2024-04-25T00:28:24.174005+0000","last_undegraded":"2024-04-25T00:28:24.174005+0000","last_fullsized":"2024-04-25T00:28:24.174005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_clean_scrub_stamp":"2024-04-25T00:23:42.305887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:44.337443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354449+0000","last_change":"2024-04-25T00:28:24.183078+0000","last_active":"2024-04-25T00:28:26.354449+0000","last_peered":"2024-04-25T00:28:26.354449+0000","last_clean":"2024-04-25T00:28:26.354449+0000","last_became_active":"2024-04-25T00:28:24.182657+0000","last_became_peered":"2024-04-25T00:28:24.182657+0000","last_unstale":"2024-04-25T00:28:26.354449+0000","last_undegraded":"2024-04-25T00:28:26.354449+0000","last_fullsized":"2024-04-25T00:28:26.354449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:14.989908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.174263+0000","last_change":"2024-04-25T00:28:23.404002+0000","last_active":"2024-04-25T00:28:24.174263+0000","last_peered":"2024-04-25T00:28:24.174263+0000","last_clean":"2024-04-25T00:28:24.174263+0000","last_became_active":"2024-04-25T00:28:23.403659+0000","last_became_peered":"2024-04-25T00:28:23.403659+0000","last_unstale":"2024-04-25T00:28:24.174263+0000","last_undegraded":"2024-04-25T00:28:24.174263+0000","last_fullsized":"2024-04-25T00:28:24.174263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:26.994242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161551+0000","last_change":"2024-04-25T00:28:23.405525+0000","last_active":"2024-04-25T00:28:25.161551+0000","last_peered":"2024-04-25T00:28:25.161551+0000","last_clean":"2024-04-25T00:28:25.161551+0000","last_became_active":"2024-04-25T00:28:23.405228+0000","last_became_peered":"2024-04-25T00:28:23.405228+0000","last_unstale":"2024-04-25T00:28:25.161551+0000","last_undegraded":"2024-04-25T00:28:25.161551+0000","last_fullsized":"2024-04-25T00:28:25.161551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:00.819210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354771+0000","last_change":"2024-04-25T00:28:24.182350+0000","last_active":"2024-04-25T00:28:26.354771+0000","last_peered":"2024-04-25T00:28:26.354771+0000","last_clean":"2024-04-25T00:28:26.354771+0000","last_became_active":"2024-04-25T00:28:24.182002+0000","last_became_peered":"2024-04-25T00:28:24.182002+0000","last_unstale":"2024-04-25T00:28:26.354771+0000","last_undegraded":"2024-04-25T00:28:26.354771+0000","last_fullsized":"2024-04-25T00:28:26.354771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:55.891046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411359+0000","last_change":"2024-04-25T00:28:23.405357+0000","last_active":"2024-04-25T00:28:25.411359+0000","last_peered":"2024-04-25T00:28:25.411359+0000","last_clean":"2024-04-25T00:28:25.411359+0000","last_became_active":"2024-04-25T00:28:23.405031+0000","last_became_peered":"2024-04-25T00:28:23.405031+0000","last_unstale":"2024-04-25T00:28:25.411359+0000","last_undegraded":"2024-04-25T00:28:25.411359+0000","last_fullsized":"2024-04-25T00:28:25.411359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:10.683462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414385+0000","last_change":"2024-04-25T00:28:24.177859+0000","last_active":"2024-04-25T00:28:25.414385+0000","last_peered":"2024-04-25T00:28:25.414385+0000","last_clean":"2024-04-25T00:28:25.414385+0000","last_became_active":"2024-04-25T00:28:24.177533+0000","last_became_peered":"2024-04-25T00:28:24.177533+0000","last_unstale":"2024-04-25T00:28:25.414385+0000","last_undegraded":"2024-04-25T00:28:25.414385+0000","last_fullsized":"2024-04-25T00:28:25.414385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:46.964934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433785+0000","last_change":"2024-04-25T00:28:23.409487+0000","last_active":"2024-04-25T00:28:26.433785+0000","last_peered":"2024-04-25T00:28:26.433785+0000","last_clean":"2024-04-25T00:28:26.433785+0000","last_became_active":"2024-04-25T00:28:23.409146+0000","last_became_peered":"2024-04-25T00:28:23.409146+0000","last_unstale":"2024-04-25T00:28:26.433785+0000","last_undegraded":"2024-04-25T00:28:26.433785+0000","last_fullsized":"2024-04-25T00:28:26.433785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.591409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160340+0000","last_change":"2024-04-25T00:28:24.185083+0000","last_active":"2024-04-25T00:28:25.160340+0000","last_peered":"2024-04-25T00:28:25.160340+0000","last_clean":"2024-04-25T00:28:25.160340+0000","last_became_active":"2024-04-25T00:28:24.184752+0000","last_became_peered":"2024-04-25T00:28:24.184752+0000","last_unstale":"2024-04-25T00:28:25.160340+0000","last_undegraded":"2024-04-25T00:28:25.160340+0000","last_fullsized":"2024-04-25T00:28:25.160340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:08.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.434198+0000","last_change":"2024-04-25T00:28:23.405662+0000","last_active":"2024-04-25T00:28:26.434198+0000","last_peered":"2024-04-25T00:28:26.434198+0000","last_clean":"2024-04-25T00:28:26.434198+0000","last_became_active":"2024-04-25T00:28:23.404949+0000","last_became_peered":"2024-04-25T00:28:23.404949+0000","last_unstale":"2024-04-25T00:28:26.434198+0000","last_undegraded":"2024-04-25T00:28:26.434198+0000","last_fullsized":"2024-04-25T00:28:26.434198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:01.989041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161234+0000","last_change":"2024-04-25T00:28:24.181404+0000","last_active":"2024-04-25T00:28:25.161234+0000","last_peered":"2024-04-25T00:28:25.161234+0000","last_clean":"2024-04-25T00:28:25.161234+0000","last_became_active":"2024-04-25T00:28:24.180399+0000","last_became_peered":"2024-04-25T00:28:24.180399+0000","last_unstale":"2024-04-25T00:28:25.161234+0000","last_undegraded":"2024-04-25T00:28:25.161234+0000","last_fullsized":"2024-04-25T00:28:25.161234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:41.955204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.434284+0000","last_change":"2024-04-25T00:28:23.405288+0000","last_active":"2024-04-25T00:28:26.434284+0000","last_peered":"2024-04-25T00:28:26.434284+0000","last_clean":"2024-04-25T00:28:26.434284+0000","last_became_active":"2024-04-25T00:28:23.404547+0000","last_became_peered":"2024-04-25T00:28:23.404547+0000","last_unstale":"2024-04-25T00:28:26.434284+0000","last_undegraded":"2024-04-25T00:28:26.434284+0000","last_fullsized":"2024-04-25T00:28:26.434284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:45.180422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392336+0000","last_change":"2024-04-25T00:28:24.176840+0000","last_active":"2024-04-25T00:28:25.392336+0000","last_peered":"2024-04-25T00:28:25.392336+0000","last_clean":"2024-04-25T00:28:25.392336+0000","last_became_active":"2024-04-25T00:28:24.176478+0000","last_became_peered":"2024-04-25T00:28:24.176478+0000","last_unstale":"2024-04-25T00:28:25.392336+0000","last_undegraded":"2024-04-25T00:28:25.392336+0000","last_fullsized":"2024-04-25T00:28:25.392336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:50.057929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.168809+0000","last_change":"2024-04-25T00:28:23.400594+0000","last_active":"2024-04-25T00:28:24.168809+0000","last_peered":"2024-04-25T00:28:24.168809+0000","last_clean":"2024-04-25T00:28:24.168809+0000","last_became_active":"2024-04-25T00:28:23.400216+0000","last_became_peered":"2024-04-25T00:28:23.400216+0000","last_unstale":"2024-04-25T00:28:24.168809+0000","last_undegraded":"2024-04-25T00:28:24.168809+0000","last_fullsized":"2024-04-25T00:28:24.168809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:18.771989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414191+0000","last_change":"2024-04-25T00:28:24.176402+0000","last_active":"2024-04-25T00:28:25.414191+0000","last_peered":"2024-04-25T00:28:25.414191+0000","last_clean":"2024-04-25T00:28:25.414191+0000","last_became_active":"2024-04-25T00:28:24.175822+0000","last_became_peered":"2024-04-25T00:28:24.175822+0000","last_unstale":"2024-04-25T00:28:25.414191+0000","last_undegraded":"2024-04-25T00:28:25.414191+0000","last_fullsized":"2024-04-25T00:28:25.414191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.952116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435131+0000","last_change":"2024-04-25T00:28:23.401158+0000","last_active":"2024-04-25T00:28:26.435131+0000","last_peered":"2024-04-25T00:28:26.435131+0000","last_clean":"2024-04-25T00:28:26.435131+0000","last_became_active":"2024-04-25T00:28:23.400819+0000","last_became_peered":"2024-04-25T00:28:23.400819+0000","last_unstale":"2024-04-25T00:28:26.435131+0000","last_undegraded":"2024-04-25T00:28:26.435131+0000","last_fullsized":"2024-04-25T00:28:26.435131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.209636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.191258+0000","last_change":"2024-04-25T00:28:24.189784+0000","last_active":"2024-04-25T00:28:24.191258+0000","last_peered":"2024-04-25T00:28:24.191258+0000","last_clean":"2024-04-25T00:28:24.191258+0000","last_became_active":"2024-04-25T00:28:24.188161+0000","last_became_peered":"2024-04-25T00:28:24.188161+0000","last_unstale":"2024-04-25T00:28:24.191258+0000","last_undegraded":"2024-04-25T00:28:24.191258+0000","last_fullsized":"2024-04-25T00:28:24.191258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:19.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.165763+0000","last_change":"2024-04-25T00:28:23.410684+0000","last_active":"2024-04-25T00:28:24.165763+0000","last_peered":"2024-04-25T00:28:24.165763+0000","last_clean":"2024-04-25T00:28:24.165763+0000","last_became_active":"2024-04-25T00:28:23.410345+0000","last_became_peered":"2024-04-25T00:28:23.410345+0000","last_unstale":"2024-04-25T00:28:24.165763+0000","last_undegraded":"2024-04-25T00:28:24.165763+0000","last_fullsized":"2024-04-25T00:28:24.165763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.190456+0000","last_change":"2024-04-25T00:28:24.189170+0000","last_active":"2024-04-25T00:28:24.190456+0000","last_peered":"2024-04-25T00:28:24.190456+0000","last_clean":"2024-04-25T00:28:24.190456+0000","last_became_active":"2024-04-25T00:28:24.188329+0000","last_became_peered":"2024-04-25T00:28:24.188329+0000","last_unstale":"2024-04-25T00:28:24.190456+0000","last_undegraded":"2024-04-25T00:28:24.190456+0000","last_fullsized":"2024-04-25T00:28:24.190456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:57.976154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354766+0000","last_change":"2024-04-25T00:28:23.405933+0000","last_active":"2024-04-25T00:28:26.354766+0000","last_peered":"2024-04-25T00:28:26.354766+0000","last_clean":"2024-04-25T00:28:26.354766+0000","last_became_active":"2024-04-25T00:28:23.405585+0000","last_became_peered":"2024-04-25T00:28:23.405585+0000","last_unstale":"2024-04-25T00:28:26.354766+0000","last_undegraded":"2024-04-25T00:28:26.354766+0000","last_fullsized":"2024-04-25T00:28:26.354766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:05.914735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.179243+0000","last_change":"2024-04-25T00:28:24.178720+0000","last_active":"2024-04-25T00:28:24.179243+0000","last_peered":"2024-04-25T00:28:24.179243+0000","last_clean":"2024-04-25T00:28:24.179243+0000","last_became_active":"2024-04-25T00:28:24.178270+0000","last_became_peered":"2024-04-25T00:28:24.178270+0000","last_unstale":"2024-04-25T00:28:24.179243+0000","last_undegraded":"2024-04-25T00:28:24.179243+0000","last_fullsized":"2024-04-25T00:28:24.179243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:16.788075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354736+0000","last_change":"2024-04-25T00:28:23.403017+0000","last_active":"2024-04-25T00:28:26.354736+0000","last_peered":"2024-04-25T00:28:26.354736+0000","last_clean":"2024-04-25T00:28:26.354736+0000","last_became_active":"2024-04-25T00:28:23.402706+0000","last_became_peered":"2024-04-25T00:28:23.402706+0000","last_unstale":"2024-04-25T00:28:26.354736+0000","last_undegraded":"2024-04-25T00:28:26.354736+0000","last_fullsized":"2024-04-25T00:28:26.354736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:06.916509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.186615+0000","last_change":"2024-04-25T00:28:24.181585+0000","last_active":"2024-04-25T00:28:24.186615+0000","last_peered":"2024-04-25T00:28:24.186615+0000","last_clean":"2024-04-25T00:28:24.186615+0000","last_became_active":"2024-04-25T00:28:24.180755+0000","last_became_peered":"2024-04-25T00:28:24.180755+0000","last_unstale":"2024-04-25T00:28:24.186615+0000","last_undegraded":"2024-04-25T00:28:24.186615+0000","last_fullsized":"2024-04-25T00:28:24.186615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:42.869814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392418+0000","last_change":"2024-04-25T00:28:24.175444+0000","last_active":"2024-04-25T00:28:25.392418+0000","last_peered":"2024-04-25T00:28:25.392418+0000","last_clean":"2024-04-25T00:28:25.392418+0000","last_became_active":"2024-04-25T00:28:24.174322+0000","last_became_peered":"2024-04-25T00:28:24.174322+0000","last_unstale":"2024-04-25T00:28:25.392418+0000","last_undegraded":"2024-04-25T00:28:25.392418+0000","last_fullsized":"2024-04-25T00:28:25.392418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:15.325122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.189100+0000","last_change":"2024-04-25T00:28:24.188603+0000","last_active":"2024-04-25T00:28:24.189100+0000","last_peered":"2024-04-25T00:28:24.189100+0000","last_clean":"2024-04-25T00:28:24.189100+0000","last_became_active":"2024-04-25T00:28:24.188300+0000","last_became_peered":"2024-04-25T00:28:24.188300+0000","last_unstale":"2024-04-25T00:28:24.189100+0000","last_undegraded":"2024-04-25T00:28:24.189100+0000","last_fullsized":"2024-04-25T00:28:24.189100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:39.221786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.184900+0000","last_change":"2024-04-25T00:28:24.183907+0000","last_active":"2024-04-25T00:28:24.184900+0000","last_peered":"2024-04-25T00:28:24.184900+0000","last_clean":"2024-04-25T00:28:24.184900+0000","last_became_active":"2024-04-25T00:28:24.183545+0000","last_became_peered":"2024-04-25T00:28:24.183545+0000","last_unstale":"2024-04-25T00:28:24.184900+0000","last_undegraded":"2024-04-25T00:28:24.184900+0000","last_fullsized":"2024-04-25T00:28:24.184900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:07.606815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160542+0000","last_change":"2024-04-25T00:28:24.178516+0000","last_active":"2024-04-25T00:28:25.160542+0000","last_peered":"2024-04-25T00:28:25.160542+0000","last_clean":"2024-04-25T00:28:25.160542+0000","last_became_active":"2024-04-25T00:28:24.177995+0000","last_became_peered":"2024-04-25T00:28:24.177995+0000","last_unstale":"2024-04-25T00:28:25.160542+0000","last_undegraded":"2024-04-25T00:28:25.160542+0000","last_fullsized":"2024-04-25T00:28:25.160542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:03.396880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354817+0000","last_change":"2024-04-25T00:28:24.175401+0000","last_active":"2024-04-25T00:28:26.354817+0000","last_peered":"2024-04-25T00:28:26.354817+0000","last_clean":"2024-04-25T00:28:26.354817+0000","last_became_active":"2024-04-25T00:28:24.174648+0000","last_became_peered":"2024-04-25T00:28:24.174648+0000","last_unstale":"2024-04-25T00:28:26.354817+0000","last_undegraded":"2024-04-25T00:28:26.354817+0000","last_fullsized":"2024-04-25T00:28:26.354817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:00.630810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392578+0000","last_change":"2024-04-25T00:28:24.173542+0000","last_active":"2024-04-25T00:28:25.392578+0000","last_peered":"2024-04-25T00:28:25.392578+0000","last_clean":"2024-04-25T00:28:25.392578+0000","last_became_active":"2024-04-25T00:28:24.173240+0000","last_became_peered":"2024-04-25T00:28:24.173240+0000","last_unstale":"2024-04-25T00:28:25.392578+0000","last_undegraded":"2024-04-25T00:28:25.392578+0000","last_fullsized":"2024-04-25T00:28:25.392578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:00.189612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.186857+0000","last_change":"2024-04-25T00:28:24.185104+0000","last_active":"2024-04-25T00:28:24.186857+0000","last_peered":"2024-04-25T00:28:24.186857+0000","last_clean":"2024-04-25T00:28:24.186857+0000","last_became_active":"2024-04-25T00:28:24.184566+0000","last_became_peered":"2024-04-25T00:28:24.184566+0000","last_unstale":"2024-04-25T00:28:24.186857+0000","last_undegraded":"2024-04-25T00:28:24.186857+0000","last_fullsized":"2024-04-25T00:28:24.186857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:13.287970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414302+0000","last_change":"2024-04-25T00:28:24.181286+0000","last_active":"2024-04-25T00:28:25.414302+0000","last_peered":"2024-04-25T00:28:25.414302+0000","last_clean":"2024-04-25T00:28:25.414302+0000","last_became_active":"2024-04-25T00:28:24.180406+0000","last_became_peered":"2024-04-25T00:28:24.180406+0000","last_unstale":"2024-04-25T00:28:25.414302+0000","last_undegraded":"2024-04-25T00:28:25.414302+0000","last_fullsized":"2024-04-25T00:28:25.414302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.644228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354437+0000","last_change":"2024-04-25T00:28:24.169883+0000","last_active":"2024-04-25T00:28:26.354437+0000","last_peered":"2024-04-25T00:28:26.354437+0000","last_clean":"2024-04-25T00:28:26.354437+0000","last_became_active":"2024-04-25T00:28:24.169558+0000","last_became_peered":"2024-04-25T00:28:24.169558+0000","last_unstale":"2024-04-25T00:28:26.354437+0000","last_undegraded":"2024-04-25T00:28:26.354437+0000","last_fullsized":"2024-04-25T00:28:26.354437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:35.615978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.413793+0000","last_change":"2024-04-25T00:28:24.180521+0000","last_active":"2024-04-25T00:28:25.413793+0000","last_peered":"2024-04-25T00:28:25.413793+0000","last_clean":"2024-04-25T00:28:25.413793+0000","last_became_active":"2024-04-25T00:28:24.179867+0000","last_became_peered":"2024-04-25T00:28:24.179867+0000","last_unstale":"2024-04-25T00:28:25.413793+0000","last_undegraded":"2024-04-25T00:28:25.413793+0000","last_fullsized":"2024-04-25T00:28:25.413793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:31.660585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354530+0000","last_change":"2024-04-25T00:28:24.185050+0000","last_active":"2024-04-25T00:28:26.354530+0000","last_peered":"2024-04-25T00:28:26.354530+0000","last_clean":"2024-04-25T00:28:26.354530+0000","last_became_active":"2024-04-25T00:28:24.184663+0000","last_became_peered":"2024-04-25T00:28:24.184663+0000","last_unstale":"2024-04-25T00:28:26.354530+0000","last_undegraded":"2024-04-25T00:28:26.354530+0000","last_fullsized":"2024-04-25T00:28:26.354530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:32.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354677+0000","last_change":"2024-04-25T00:28:24.177948+0000","last_active":"2024-04-25T00:28:26.354677+0000","last_peered":"2024-04-25T00:28:26.354677+0000","last_clean":"2024-04-25T00:28:26.354677+0000","last_became_active":"2024-04-25T00:28:24.177627+0000","last_became_peered":"2024-04-25T00:28:24.177627+0000","last_unstale":"2024-04-25T00:28:26.354677+0000","last_undegraded":"2024-04-25T00:28:26.354677+0000","last_fullsized":"2024-04-25T00:28:26.354677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:29.128868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392492+0000","last_change":"2024-04-25T00:28:24.176115+0000","last_active":"2024-04-25T00:28:25.392492+0000","last_peered":"2024-04-25T00:28:25.392492+0000","last_clean":"2024-04-25T00:28:25.392492+0000","last_became_active":"2024-04-25T00:28:24.175735+0000","last_became_peered":"2024-04-25T00:28:24.175735+0000","last_unstale":"2024-04-25T00:28:25.392492+0000","last_undegraded":"2024-04-25T00:28:25.392492+0000","last_fullsized":"2024-04-25T00:28:25.392492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:58.595542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280907,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":98,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":13648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":447512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":1039866,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":447714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266722,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":970752,"data_stored":1028728,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":447020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":448542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":440006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":447034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":447574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1028910,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:28:30.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 -- 172.21.15.31:0/570362071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408102200 msgr2=0x7fc4081045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/570362071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408102200 0x7fc4081045f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009900 tx=0x7fc3f802a410 comp rx=0 tx=0).stop 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 -- 172.21.15.31:0/570362071 shutdown_connections 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/570362071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc408107460 0x7fc4081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/570362071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc408104b30 0x7fc408106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/570362071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408102200 0x7fc4081045f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 -- 172.21.15.31:0/570362071 >> 172.21.15.31:0/570362071 conn(0x7fc4080fbbf0 msgr2=0x7fc4080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.110+0000 7fc40e6fc640 1 -- 172.21.15.31:0/570362071 shutdown_connections 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 -- 172.21.15.31:0/570362071 wait complete. 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 Processor -- start 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 -- start start 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc408102200 0x7fc408101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc408104b30 0x7fc4081001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 0x7fc4081006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d2e0 con 0x7fc408107460 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d160 con 0x7fc408102200 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.111+0000 7fc40e6fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d460 con 0x7fc408104b30 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 0x7fc4081006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 0x7fc4081006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50296/0 (socket says 172.21.15.31:50296) 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 -- 172.21.15.31:0/3490377165 learned_addr learned my addr 172.21.15.31:0/3490377165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc4077fe640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc408104b30 0x7fc4081001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc407fff640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc408102200 0x7fc408101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 -- 172.21.15.31:0/3490377165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc408104b30 msgr2=0x7fc4081001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc408104b30 0x7fc4081001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 -- 172.21.15.31:0/3490377165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc408102200 msgr2=0x7fc408101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc408102200 0x7fc408101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 -- 172.21.15.31:0/3490377165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc408100f80 con 0x7fc408107460 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40cc99640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 0x7fc4081006f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc40000cce0 tx=0x7fc40000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000f450 con 0x7fc408107460 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40000faa0 con 0x7fc408107460 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4081a1b00 con 0x7fc408107460 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400018b70 con 0x7fc408107460 2024-04-25T00:28:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.112+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a2050 con 0x7fc408107460 2024-04-25T00:28:31.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.113+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4080fd6e0 con 0x7fc408107460 2024-04-25T00:28:31.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.114+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc400018d10 con 0x7fc408107460 2024-04-25T00:28:31.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.115+0000 7fc4057fa640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.115+0000 7fc407fff640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.115+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc4000cf540 con 0x7fc408107460 2024-04-25T00:28:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.116+0000 7fc407fff640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006010 tx=0x7fc3f8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.119+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc400014030 con 0x7fc408107460 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] up:active 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:28:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4071190719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:31.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:31 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/162358812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:31.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.303+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc408061b80 con 0x7fc3f00a2ca0 2024-04-25T00:28:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.311+0000 7fc4057fa640 1 -- 172.21.15.31:0/3490377165 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258959 (secure 0 0 0) 0x7fc408061b80 con 0x7fc3f00a2ca0 2024-04-25T00:28:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:31.316 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T00:28:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.313+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc3f00a2ca0 msgr2=0x7fc3f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.313+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006010 tx=0x7fc3f8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 msgr2=0x7fc4081006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 0x7fc4081006f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc40000cce0 tx=0x7fc40000a620 comp rx=0 tx=0).stop 2024-04-25T00:28:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 shutdown_connections 2024-04-25T00:28:31.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc408107460 0x7fc4081006f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc408104b30 0x7fc4081001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 --2- 172.21.15.31:0/3490377165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc408102200 0x7fc408101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 >> 172.21.15.31:0/3490377165 conn(0x7fc4080fbbf0 msgr2=0x7fc408102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 shutdown_connections 2024-04-25T00:28:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:31.314+0000 7fc40e6fc640 1 -- 172.21.15.31:0/3490377165 wait complete. 2024-04-25T00:28:31.325 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-04-25T00:28:31.302865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595713,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":127,"num_read_kb":110,"num_write":293,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360424,"kb_used_data":11240,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581400,"statfs":{"total":1151940427776,"available":1151571353600,"internally_reserved":0,"allocated":11509760,"data_stored":7130189,"data_compressed":107184,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5345,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":112,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.918807"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392588+0000","last_change":"2024-04-25T00:28:24.187326+0000","last_active":"2024-04-25T00:28:25.392588+0000","last_peered":"2024-04-25T00:28:25.392588+0000","last_clean":"2024-04-25T00:28:25.392588+0000","last_became_active":"2024-04-25T00:28:24.186536+0000","last_became_peered":"2024-04-25T00:28:24.186536+0000","last_unstale":"2024-04-25T00:28:25.392588+0000","last_undegraded":"2024-04-25T00:28:25.392588+0000","last_fullsized":"2024-04-25T00:28:25.392588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:25.035523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160508+0000","last_change":"2024-04-25T00:28:24.181763+0000","last_active":"2024-04-25T00:28:25.160508+0000","last_peered":"2024-04-25T00:28:25.160508+0000","last_clean":"2024-04-25T00:28:25.160508+0000","last_became_active":"2024-04-25T00:28:24.181437+0000","last_became_peered":"2024-04-25T00:28:24.181437+0000","last_unstale":"2024-04-25T00:28:25.160508+0000","last_undegraded":"2024-04-25T00:28:25.160508+0000","last_fullsized":"2024-04-25T00:28:25.160508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:54.019249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160575+0000","last_change":"2024-04-25T00:28:24.186648+0000","last_active":"2024-04-25T00:28:25.160575+0000","last_peered":"2024-04-25T00:28:25.160575+0000","last_clean":"2024-04-25T00:28:25.160575+0000","last_became_active":"2024-04-25T00:28:24.186303+0000","last_became_peered":"2024-04-25T00:28:24.186303+0000","last_unstale":"2024-04-25T00:28:25.160575+0000","last_undegraded":"2024-04-25T00:28:25.160575+0000","last_fullsized":"2024-04-25T00:28:25.160575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:14.521970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161448+0000","last_change":"2024-04-25T00:28:24.179138+0000","last_active":"2024-04-25T00:28:25.161448+0000","last_peered":"2024-04-25T00:28:25.161448+0000","last_clean":"2024-04-25T00:28:25.161448+0000","last_became_active":"2024-04-25T00:28:24.178664+0000","last_became_peered":"2024-04-25T00:28:24.178664+0000","last_unstale":"2024-04-25T00:28:25.161448+0000","last_undegraded":"2024-04-25T00:28:25.161448+0000","last_fullsized":"2024-04-25T00:28:25.161448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:07.869662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161400+0000","last_change":"2024-04-25T00:28:24.174081+0000","last_active":"2024-04-25T00:28:25.161400+0000","last_peered":"2024-04-25T00:28:25.161400+0000","last_clean":"2024-04-25T00:28:25.161400+0000","last_became_active":"2024-04-25T00:28:24.173670+0000","last_became_peered":"2024-04-25T00:28:24.173670+0000","last_unstale":"2024-04-25T00:28:25.161400+0000","last_undegraded":"2024-04-25T00:28:25.161400+0000","last_fullsized":"2024-04-25T00:28:25.161400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:43.255598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355757+0000","last_change":"2024-04-25T00:28:24.174664+0000","last_active":"2024-04-25T00:28:26.355757+0000","last_peered":"2024-04-25T00:28:26.355757+0000","last_clean":"2024-04-25T00:28:26.355757+0000","last_became_active":"2024-04-25T00:28:24.174238+0000","last_became_peered":"2024-04-25T00:28:24.174238+0000","last_unstale":"2024-04-25T00:28:26.355757+0000","last_undegraded":"2024-04-25T00:28:26.355757+0000","last_fullsized":"2024-04-25T00:28:26.355757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:21.214976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354351+0000","last_change":"2024-04-25T00:28:24.181123+0000","last_active":"2024-04-25T00:28:26.354351+0000","last_peered":"2024-04-25T00:28:26.354351+0000","last_clean":"2024-04-25T00:28:26.354351+0000","last_became_active":"2024-04-25T00:28:24.179516+0000","last_became_peered":"2024-04-25T00:28:24.179516+0000","last_unstale":"2024-04-25T00:28:26.354351+0000","last_undegraded":"2024-04-25T00:28:26.354351+0000","last_fullsized":"2024-04-25T00:28:26.354351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:11.822968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.413755+0000","last_change":"2024-04-25T00:28:24.179964+0000","last_active":"2024-04-25T00:28:25.413755+0000","last_peered":"2024-04-25T00:28:25.413755+0000","last_clean":"2024-04-25T00:28:25.413755+0000","last_became_active":"2024-04-25T00:28:24.179585+0000","last_became_peered":"2024-04-25T00:28:24.179585+0000","last_unstale":"2024-04-25T00:28:25.413755+0000","last_undegraded":"2024-04-25T00:28:25.413755+0000","last_fullsized":"2024-04-25T00:28:25.413755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:42.990145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.356167+0000","last_change":"2024-04-25T00:28:24.170818+0000","last_active":"2024-04-25T00:28:26.356167+0000","last_peered":"2024-04-25T00:28:26.356167+0000","last_clean":"2024-04-25T00:28:26.356167+0000","last_became_active":"2024-04-25T00:28:24.170412+0000","last_became_peered":"2024-04-25T00:28:24.170412+0000","last_unstale":"2024-04-25T00:28:26.356167+0000","last_undegraded":"2024-04-25T00:28:26.356167+0000","last_fullsized":"2024-04-25T00:28:26.356167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:20.456317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355533+0000","last_change":"2024-04-25T00:28:24.182509+0000","last_active":"2024-04-25T00:28:26.355533+0000","last_peered":"2024-04-25T00:28:26.355533+0000","last_clean":"2024-04-25T00:28:26.355533+0000","last_became_active":"2024-04-25T00:28:24.182141+0000","last_became_peered":"2024-04-25T00:28:24.182141+0000","last_unstale":"2024-04-25T00:28:26.355533+0000","last_undegraded":"2024-04-25T00:28:26.355533+0000","last_fullsized":"2024-04-25T00:28:26.355533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:33.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355793+0000","last_change":"2024-04-25T00:28:24.189265+0000","last_active":"2024-04-25T00:28:26.355793+0000","last_peered":"2024-04-25T00:28:26.355793+0000","last_clean":"2024-04-25T00:28:26.355793+0000","last_became_active":"2024-04-25T00:28:24.188829+0000","last_became_peered":"2024-04-25T00:28:24.188829+0000","last_unstale":"2024-04-25T00:28:26.355793+0000","last_undegraded":"2024-04-25T00:28:26.355793+0000","last_fullsized":"2024-04-25T00:28:26.355793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:33.930373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161340+0000","last_change":"2024-04-25T00:28:24.180145+0000","last_active":"2024-04-25T00:28:25.161340+0000","last_peered":"2024-04-25T00:28:25.161340+0000","last_clean":"2024-04-25T00:28:25.161340+0000","last_became_active":"2024-04-25T00:28:24.179802+0000","last_became_peered":"2024-04-25T00:28:24.179802+0000","last_unstale":"2024-04-25T00:28:25.161340+0000","last_undegraded":"2024-04-25T00:28:25.161340+0000","last_fullsized":"2024-04-25T00:28:25.161340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:34.367575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354937+0000","last_change":"2024-04-25T00:28:24.183769+0000","last_active":"2024-04-25T00:28:26.354937+0000","last_peered":"2024-04-25T00:28:26.354937+0000","last_clean":"2024-04-25T00:28:26.354937+0000","last_became_active":"2024-04-25T00:28:24.183432+0000","last_became_peered":"2024-04-25T00:28:24.183432+0000","last_unstale":"2024-04-25T00:28:26.354937+0000","last_undegraded":"2024-04-25T00:28:26.354937+0000","last_fullsized":"2024-04-25T00:28:26.354937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:27.656794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160247+0000","last_change":"2024-04-25T00:28:24.184364+0000","last_active":"2024-04-25T00:28:25.160247+0000","last_peered":"2024-04-25T00:28:25.160247+0000","last_clean":"2024-04-25T00:28:25.160247+0000","last_became_active":"2024-04-25T00:28:24.184038+0000","last_became_peered":"2024-04-25T00:28:24.184038+0000","last_unstale":"2024-04-25T00:28:25.160247+0000","last_undegraded":"2024-04-25T00:28:25.160247+0000","last_fullsized":"2024-04-25T00:28:25.160247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:50.495466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160091+0000","last_change":"2024-04-25T00:28:24.180236+0000","last_active":"2024-04-25T00:28:25.160091+0000","last_peered":"2024-04-25T00:28:25.160091+0000","last_clean":"2024-04-25T00:28:25.160091+0000","last_became_active":"2024-04-25T00:28:24.179929+0000","last_became_peered":"2024-04-25T00:28:24.179929+0000","last_unstale":"2024-04-25T00:28:25.160091+0000","last_undegraded":"2024-04-25T00:28:25.160091+0000","last_fullsized":"2024-04-25T00:28:25.160091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:29.858564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354243+0000","last_change":"2024-04-25T00:28:24.185605+0000","last_active":"2024-04-25T00:28:26.354243+0000","last_peered":"2024-04-25T00:28:26.354243+0000","last_clean":"2024-04-25T00:28:26.354243+0000","last_became_active":"2024-04-25T00:28:24.185270+0000","last_became_peered":"2024-04-25T00:28:24.185270+0000","last_unstale":"2024-04-25T00:28:26.354243+0000","last_undegraded":"2024-04-25T00:28:26.354243+0000","last_fullsized":"2024-04-25T00:28:26.354243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:58.798429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392710+0000","last_change":"2024-04-25T00:28:24.187625+0000","last_active":"2024-04-25T00:28:25.392710+0000","last_peered":"2024-04-25T00:28:25.392710+0000","last_clean":"2024-04-25T00:28:25.392710+0000","last_became_active":"2024-04-25T00:28:24.186961+0000","last_became_peered":"2024-04-25T00:28:24.186961+0000","last_unstale":"2024-04-25T00:28:25.392710+0000","last_undegraded":"2024-04-25T00:28:25.392710+0000","last_fullsized":"2024-04-25T00:28:25.392710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:26.099099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411375+0000","last_change":"2024-04-25T00:28:24.177342+0000","last_active":"2024-04-25T00:28:25.411375+0000","last_peered":"2024-04-25T00:28:25.411375+0000","last_clean":"2024-04-25T00:28:25.411375+0000","last_became_active":"2024-04-25T00:28:24.176763+0000","last_became_peered":"2024-04-25T00:28:24.176763+0000","last_unstale":"2024-04-25T00:28:25.411375+0000","last_undegraded":"2024-04-25T00:28:25.411375+0000","last_fullsized":"2024-04-25T00:28:25.411375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:47.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.187348+0000","last_change":"2024-04-25T00:28:24.186069+0000","last_active":"2024-04-25T00:28:24.187348+0000","last_peered":"2024-04-25T00:28:24.187348+0000","last_clean":"2024-04-25T00:28:24.187348+0000","last_became_active":"2024-04-25T00:28:24.185713+0000","last_became_peered":"2024-04-25T00:28:24.185713+0000","last_unstale":"2024-04-25T00:28:24.187348+0000","last_undegraded":"2024-04-25T00:28:24.187348+0000","last_fullsized":"2024-04-25T00:28:24.187348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:21.862359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411258+0000","last_change":"2024-04-25T00:28:23.402187+0000","last_active":"2024-04-25T00:28:25.411258+0000","last_peered":"2024-04-25T00:28:25.411258+0000","last_clean":"2024-04-25T00:28:25.411258+0000","last_became_active":"2024-04-25T00:28:23.401868+0000","last_became_peered":"2024-04-25T00:28:23.401868+0000","last_unstale":"2024-04-25T00:28:25.411258+0000","last_undegraded":"2024-04-25T00:28:25.411258+0000","last_fullsized":"2024-04-25T00:28:25.411258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:44.022628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414232+0000","last_change":"2024-04-25T00:28:24.180731+0000","last_active":"2024-04-25T00:28:25.414232+0000","last_peered":"2024-04-25T00:28:25.414232+0000","last_clean":"2024-04-25T00:28:25.414232+0000","last_became_active":"2024-04-25T00:28:24.180274+0000","last_became_peered":"2024-04-25T00:28:24.180274+0000","last_unstale":"2024-04-25T00:28:25.414232+0000","last_undegraded":"2024-04-25T00:28:25.414232+0000","last_fullsized":"2024-04-25T00:28:25.414232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:46.102326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414196+0000","last_change":"2024-04-25T00:28:23.405683+0000","last_active":"2024-04-25T00:28:25.414196+0000","last_peered":"2024-04-25T00:28:25.414196+0000","last_clean":"2024-04-25T00:28:25.414196+0000","last_became_active":"2024-04-25T00:28:23.405305+0000","last_became_peered":"2024-04-25T00:28:23.405305+0000","last_unstale":"2024-04-25T00:28:25.414196+0000","last_undegraded":"2024-04-25T00:28:25.414196+0000","last_fullsized":"2024-04-25T00:28:25.414196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:39.782047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160676+0000","last_change":"2024-04-25T00:28:24.187565+0000","last_active":"2024-04-25T00:28:25.160676+0000","last_peered":"2024-04-25T00:28:25.160676+0000","last_clean":"2024-04-25T00:28:25.160676+0000","last_became_active":"2024-04-25T00:28:24.186397+0000","last_became_peered":"2024-04-25T00:28:24.186397+0000","last_unstale":"2024-04-25T00:28:25.160676+0000","last_undegraded":"2024-04-25T00:28:25.160676+0000","last_fullsized":"2024-04-25T00:28:25.160676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:36.750031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.442562+0000","last_change":"2024-04-25T00:28:23.398726+0000","last_active":"2024-04-25T00:28:27.442562+0000","last_peered":"2024-04-25T00:28:27.442562+0000","last_clean":"2024-04-25T00:28:27.442562+0000","last_became_active":"2024-04-25T00:28:23.398370+0000","last_became_peered":"2024-04-25T00:28:23.398370+0000","last_unstale":"2024-04-25T00:28:27.442562+0000","last_undegraded":"2024-04-25T00:28:27.442562+0000","last_fullsized":"2024-04-25T00:28:27.442562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:19.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354678+0000","last_change":"2024-04-25T00:28:24.181424+0000","last_active":"2024-04-25T00:28:26.354678+0000","last_peered":"2024-04-25T00:28:26.354678+0000","last_clean":"2024-04-25T00:28:26.354678+0000","last_became_active":"2024-04-25T00:28:24.179077+0000","last_became_peered":"2024-04-25T00:28:24.179077+0000","last_unstale":"2024-04-25T00:28:26.354678+0000","last_undegraded":"2024-04-25T00:28:26.354678+0000","last_fullsized":"2024-04-25T00:28:26.354678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.926451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161493+0000","last_change":"2024-04-25T00:28:23.402419+0000","last_active":"2024-04-25T00:28:25.161493+0000","last_peered":"2024-04-25T00:28:25.161493+0000","last_clean":"2024-04-25T00:28:25.161493+0000","last_became_active":"2024-04-25T00:28:23.402070+0000","last_became_peered":"2024-04-25T00:28:23.402070+0000","last_unstale":"2024-04-25T00:28:25.161493+0000","last_undegraded":"2024-04-25T00:28:25.161493+0000","last_fullsized":"2024-04-25T00:28:25.161493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:08.799577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354865+0000","last_change":"2024-04-25T00:28:24.178856+0000","last_active":"2024-04-25T00:28:26.354865+0000","last_peered":"2024-04-25T00:28:26.354865+0000","last_clean":"2024-04-25T00:28:26.354865+0000","last_became_active":"2024-04-25T00:28:24.178469+0000","last_became_peered":"2024-04-25T00:28:24.178469+0000","last_unstale":"2024-04-25T00:28:26.354865+0000","last_undegraded":"2024-04-25T00:28:26.354865+0000","last_fullsized":"2024-04-25T00:28:26.354865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:28.211603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.436956+0000","last_change":"2024-04-25T00:28:23.400250+0000","last_active":"2024-04-25T00:28:26.436956+0000","last_peered":"2024-04-25T00:28:26.436956+0000","last_clean":"2024-04-25T00:28:26.436956+0000","last_became_active":"2024-04-25T00:28:23.399874+0000","last_became_peered":"2024-04-25T00:28:23.399874+0000","last_unstale":"2024-04-25T00:28:26.436956+0000","last_undegraded":"2024-04-25T00:28:26.436956+0000","last_fullsized":"2024-04-25T00:28:26.436956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:08.332441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161366+0000","last_change":"2024-04-25T00:28:24.170176+0000","last_active":"2024-04-25T00:28:25.161366+0000","last_peered":"2024-04-25T00:28:25.161366+0000","last_clean":"2024-04-25T00:28:25.161366+0000","last_became_active":"2024-04-25T00:28:24.169493+0000","last_became_peered":"2024-04-25T00:28:24.169493+0000","last_unstale":"2024-04-25T00:28:25.161366+0000","last_undegraded":"2024-04-25T00:28:25.161366+0000","last_fullsized":"2024-04-25T00:28:25.161366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:43.855777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.444321+0000","last_change":"2024-04-25T00:28:23.398003+0000","last_active":"2024-04-25T00:28:27.444321+0000","last_peered":"2024-04-25T00:28:27.444321+0000","last_clean":"2024-04-25T00:28:27.444321+0000","last_became_active":"2024-04-25T00:28:23.397229+0000","last_became_peered":"2024-04-25T00:28:23.397229+0000","last_unstale":"2024-04-25T00:28:27.444321+0000","last_undegraded":"2024-04-25T00:28:27.444321+0000","last_fullsized":"2024-04-25T00:28:27.444321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:59.475252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354752+0000","last_change":"2024-04-25T00:28:24.180899+0000","last_active":"2024-04-25T00:28:26.354752+0000","last_peered":"2024-04-25T00:28:26.354752+0000","last_clean":"2024-04-25T00:28:26.354752+0000","last_became_active":"2024-04-25T00:28:24.178987+0000","last_became_peered":"2024-04-25T00:28:24.178987+0000","last_unstale":"2024-04-25T00:28:26.354752+0000","last_undegraded":"2024-04-25T00:28:26.354752+0000","last_fullsized":"2024-04-25T00:28:26.354752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:22.561636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.443204+0000","last_change":"2024-04-25T00:28:23.397855+0000","last_active":"2024-04-25T00:28:27.443204+0000","last_peered":"2024-04-25T00:28:27.443204+0000","last_clean":"2024-04-25T00:28:27.443204+0000","last_became_active":"2024-04-25T00:28:23.397465+0000","last_became_peered":"2024-04-25T00:28:23.397465+0000","last_unstale":"2024-04-25T00:28:27.443204+0000","last_undegraded":"2024-04-25T00:28:27.443204+0000","last_fullsized":"2024-04-25T00:28:27.443204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.357397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.413711+0000","last_change":"2024-04-25T00:28:24.172890+0000","last_active":"2024-04-25T00:28:25.413711+0000","last_peered":"2024-04-25T00:28:25.413711+0000","last_clean":"2024-04-25T00:28:25.413711+0000","last_became_active":"2024-04-25T00:28:24.172250+0000","last_became_peered":"2024-04-25T00:28:24.172250+0000","last_unstale":"2024-04-25T00:28:25.413711+0000","last_undegraded":"2024-04-25T00:28:25.413711+0000","last_fullsized":"2024-04-25T00:28:25.413711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:34.372811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435327+0000","last_change":"2024-04-25T00:28:23.401140+0000","last_active":"2024-04-25T00:28:26.435327+0000","last_peered":"2024-04-25T00:28:26.435327+0000","last_clean":"2024-04-25T00:28:26.435327+0000","last_became_active":"2024-04-25T00:28:23.400804+0000","last_became_peered":"2024-04-25T00:28:23.400804+0000","last_unstale":"2024-04-25T00:28:26.435327+0000","last_undegraded":"2024-04-25T00:28:26.435327+0000","last_fullsized":"2024-04-25T00:28:26.435327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:00.044141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411401+0000","last_change":"2024-04-25T00:28:24.175596+0000","last_active":"2024-04-25T00:28:25.411401+0000","last_peered":"2024-04-25T00:28:25.411401+0000","last_clean":"2024-04-25T00:28:25.411401+0000","last_became_active":"2024-04-25T00:28:24.175085+0000","last_became_peered":"2024-04-25T00:28:24.175085+0000","last_unstale":"2024-04-25T00:28:25.411401+0000","last_undegraded":"2024-04-25T00:28:25.411401+0000","last_fullsized":"2024-04-25T00:28:25.411401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:05.801995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433879+0000","last_change":"2024-04-25T00:28:23.399427+0000","last_active":"2024-04-25T00:28:26.433879+0000","last_peered":"2024-04-25T00:28:26.433879+0000","last_clean":"2024-04-25T00:28:26.433879+0000","last_became_active":"2024-04-25T00:28:23.399067+0000","last_became_peered":"2024-04-25T00:28:23.399067+0000","last_unstale":"2024-04-25T00:28:26.433879+0000","last_undegraded":"2024-04-25T00:28:26.433879+0000","last_fullsized":"2024-04-25T00:28:26.433879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:06.853292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355993+0000","last_change":"2024-04-25T00:28:24.177367+0000","last_active":"2024-04-25T00:28:26.355993+0000","last_peered":"2024-04-25T00:28:26.355993+0000","last_clean":"2024-04-25T00:28:26.355993+0000","last_became_active":"2024-04-25T00:28:24.176890+0000","last_became_peered":"2024-04-25T00:28:24.176890+0000","last_unstale":"2024-04-25T00:28:26.355993+0000","last_undegraded":"2024-04-25T00:28:26.355993+0000","last_fullsized":"2024-04-25T00:28:26.355993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:27.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:29.174888+0000","last_change":"2024-04-25T00:28:23.409705+0000","last_active":"2024-04-25T00:28:29.174888+0000","last_peered":"2024-04-25T00:28:29.174888+0000","last_clean":"2024-04-25T00:28:29.174888+0000","last_became_active":"2024-04-25T00:28:23.408596+0000","last_became_peered":"2024-04-25T00:28:23.408596+0000","last_unstale":"2024-04-25T00:28:29.174888+0000","last_undegraded":"2024-04-25T00:28:29.174888+0000","last_fullsized":"2024-04-25T00:28:29.174888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:25.684497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160686+0000","last_change":"2024-04-25T00:28:24.179361+0000","last_active":"2024-04-25T00:28:25.160686+0000","last_peered":"2024-04-25T00:28:25.160686+0000","last_clean":"2024-04-25T00:28:25.160686+0000","last_became_active":"2024-04-25T00:28:24.179019+0000","last_became_peered":"2024-04-25T00:28:24.179019+0000","last_unstale":"2024-04-25T00:28:25.160686+0000","last_undegraded":"2024-04-25T00:28:25.160686+0000","last_fullsized":"2024-04-25T00:28:25.160686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:08.543283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433353+0000","last_change":"2024-04-25T00:28:23.404051+0000","last_active":"2024-04-25T00:28:26.433353+0000","last_peered":"2024-04-25T00:28:26.433353+0000","last_clean":"2024-04-25T00:28:26.433353+0000","last_became_active":"2024-04-25T00:28:23.403728+0000","last_became_peered":"2024-04-25T00:28:23.403728+0000","last_unstale":"2024-04-25T00:28:26.433353+0000","last_undegraded":"2024-04-25T00:28:26.433353+0000","last_fullsized":"2024-04-25T00:28:26.433353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:27.403464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355028+0000","last_change":"2024-04-25T00:28:24.178506+0000","last_active":"2024-04-25T00:28:26.355028+0000","last_peered":"2024-04-25T00:28:26.355028+0000","last_clean":"2024-04-25T00:28:26.355028+0000","last_became_active":"2024-04-25T00:28:24.177053+0000","last_became_peered":"2024-04-25T00:28:24.177053+0000","last_unstale":"2024-04-25T00:28:26.355028+0000","last_undegraded":"2024-04-25T00:28:26.355028+0000","last_fullsized":"2024-04-25T00:28:26.355028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:04.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433978+0000","last_change":"2024-04-25T00:28:23.400563+0000","last_active":"2024-04-25T00:28:26.433978+0000","last_peered":"2024-04-25T00:28:26.433978+0000","last_clean":"2024-04-25T00:28:26.433978+0000","last_became_active":"2024-04-25T00:28:23.399982+0000","last_became_peered":"2024-04-25T00:28:23.399982+0000","last_unstale":"2024-04-25T00:28:26.433978+0000","last_undegraded":"2024-04-25T00:28:26.433978+0000","last_fullsized":"2024-04-25T00:28:26.433978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:01.652027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161588+0000","last_change":"2024-04-25T00:28:24.179687+0000","last_active":"2024-04-25T00:28:25.161588+0000","last_peered":"2024-04-25T00:28:25.161588+0000","last_clean":"2024-04-25T00:28:25.161588+0000","last_became_active":"2024-04-25T00:28:24.179345+0000","last_became_peered":"2024-04-25T00:28:24.179345+0000","last_unstale":"2024-04-25T00:28:25.161588+0000","last_undegraded":"2024-04-25T00:28:25.161588+0000","last_fullsized":"2024-04-25T00:28:25.161588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:49.807417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.417344+0000","last_change":"2024-04-25T00:28:23.401353+0000","last_active":"2024-04-25T00:28:24.417344+0000","last_peered":"2024-04-25T00:28:24.417344+0000","last_clean":"2024-04-25T00:28:24.417344+0000","last_became_active":"2024-04-25T00:28:23.400985+0000","last_became_peered":"2024-04-25T00:28:23.400985+0000","last_unstale":"2024-04-25T00:28:24.417344+0000","last_undegraded":"2024-04-25T00:28:24.417344+0000","last_fullsized":"2024-04-25T00:28:24.417344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.277514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354636+0000","last_change":"2024-04-25T00:28:24.178796+0000","last_active":"2024-04-25T00:28:26.354636+0000","last_peered":"2024-04-25T00:28:26.354636+0000","last_clean":"2024-04-25T00:28:26.354636+0000","last_became_active":"2024-04-25T00:28:24.178457+0000","last_became_peered":"2024-04-25T00:28:24.178457+0000","last_unstale":"2024-04-25T00:28:26.354636+0000","last_undegraded":"2024-04-25T00:28:26.354636+0000","last_fullsized":"2024-04-25T00:28:26.354636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:48.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435499+0000","last_change":"2024-04-25T00:28:23.406046+0000","last_active":"2024-04-25T00:28:26.435499+0000","last_peered":"2024-04-25T00:28:26.435499+0000","last_clean":"2024-04-25T00:28:26.435499+0000","last_became_active":"2024-04-25T00:28:23.405686+0000","last_became_peered":"2024-04-25T00:28:23.405686+0000","last_unstale":"2024-04-25T00:28:26.435499+0000","last_undegraded":"2024-04-25T00:28:26.435499+0000","last_fullsized":"2024-04-25T00:28:26.435499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:30.551884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355445+0000","last_change":"2024-04-25T00:28:24.188251+0000","last_active":"2024-04-25T00:28:26.355445+0000","last_peered":"2024-04-25T00:28:26.355445+0000","last_clean":"2024-04-25T00:28:26.355445+0000","last_became_active":"2024-04-25T00:28:24.187876+0000","last_became_peered":"2024-04-25T00:28:24.187876+0000","last_unstale":"2024-04-25T00:28:26.355445+0000","last_undegraded":"2024-04-25T00:28:26.355445+0000","last_fullsized":"2024-04-25T00:28:26.355445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:18.269074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.441291+0000","last_change":"2024-04-25T00:28:23.407240+0000","last_active":"2024-04-25T00:28:27.441291+0000","last_peered":"2024-04-25T00:28:27.441291+0000","last_clean":"2024-04-25T00:28:27.441291+0000","last_became_active":"2024-04-25T00:28:23.406794+0000","last_became_peered":"2024-04-25T00:28:23.406794+0000","last_unstale":"2024-04-25T00:28:27.441291+0000","last_undegraded":"2024-04-25T00:28:27.441291+0000","last_fullsized":"2024-04-25T00:28:27.441291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:35.861733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354260+0000","last_change":"2024-04-25T00:28:24.183784+0000","last_active":"2024-04-25T00:28:26.354260+0000","last_peered":"2024-04-25T00:28:26.354260+0000","last_clean":"2024-04-25T00:28:26.354260+0000","last_became_active":"2024-04-25T00:28:24.183417+0000","last_became_peered":"2024-04-25T00:28:24.183417+0000","last_unstale":"2024-04-25T00:28:26.354260+0000","last_undegraded":"2024-04-25T00:28:26.354260+0000","last_fullsized":"2024-04-25T00:28:26.354260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:43.182050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.444731+0000","last_change":"2024-04-25T00:28:23.406324+0000","last_active":"2024-04-25T00:28:27.444731+0000","last_peered":"2024-04-25T00:28:27.444731+0000","last_clean":"2024-04-25T00:28:27.444731+0000","last_became_active":"2024-04-25T00:28:23.405964+0000","last_became_peered":"2024-04-25T00:28:23.405964+0000","last_unstale":"2024-04-25T00:28:27.444731+0000","last_undegraded":"2024-04-25T00:28:27.444731+0000","last_fullsized":"2024-04-25T00:28:27.444731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:17.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354614+0000","last_change":"2024-04-25T00:28:24.177898+0000","last_active":"2024-04-25T00:28:26.354614+0000","last_peered":"2024-04-25T00:28:26.354614+0000","last_clean":"2024-04-25T00:28:26.354614+0000","last_became_active":"2024-04-25T00:28:24.177578+0000","last_became_peered":"2024-04-25T00:28:24.177578+0000","last_unstale":"2024-04-25T00:28:26.354614+0000","last_undegraded":"2024-04-25T00:28:26.354614+0000","last_fullsized":"2024-04-25T00:28:26.354614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:36.294161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161599+0000","last_change":"2024-04-25T00:28:23.399570+0000","last_active":"2024-04-25T00:28:25.161599+0000","last_peered":"2024-04-25T00:28:25.161599+0000","last_clean":"2024-04-25T00:28:25.161599+0000","last_became_active":"2024-04-25T00:28:23.399230+0000","last_became_peered":"2024-04-25T00:28:23.399230+0000","last_unstale":"2024-04-25T00:28:25.161599+0000","last_undegraded":"2024-04-25T00:28:25.161599+0000","last_fullsized":"2024-04-25T00:28:25.161599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:46.850590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392493+0000","last_change":"2024-04-25T00:28:24.186113+0000","last_active":"2024-04-25T00:28:25.392493+0000","last_peered":"2024-04-25T00:28:25.392493+0000","last_clean":"2024-04-25T00:28:25.392493+0000","last_became_active":"2024-04-25T00:28:24.185743+0000","last_became_peered":"2024-04-25T00:28:24.185743+0000","last_unstale":"2024-04-25T00:28:25.392493+0000","last_undegraded":"2024-04-25T00:28:25.392493+0000","last_fullsized":"2024-04-25T00:28:25.392493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:10.973285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411322+0000","last_change":"2024-04-25T00:28:23.396054+0000","last_active":"2024-04-25T00:28:25.411322+0000","last_peered":"2024-04-25T00:28:25.411322+0000","last_clean":"2024-04-25T00:28:25.411322+0000","last_became_active":"2024-04-25T00:28:23.395707+0000","last_became_peered":"2024-04-25T00:28:23.395707+0000","last_unstale":"2024-04-25T00:28:25.411322+0000","last_undegraded":"2024-04-25T00:28:25.411322+0000","last_fullsized":"2024-04-25T00:28:25.411322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:20.169476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355165+0000","last_change":"2024-04-25T00:28:24.185655+0000","last_active":"2024-04-25T00:28:26.355165+0000","last_peered":"2024-04-25T00:28:26.355165+0000","last_clean":"2024-04-25T00:28:26.355165+0000","last_became_active":"2024-04-25T00:28:24.185123+0000","last_became_peered":"2024-04-25T00:28:24.185123+0000","last_unstale":"2024-04-25T00:28:26.355165+0000","last_undegraded":"2024-04-25T00:28:26.355165+0000","last_fullsized":"2024-04-25T00:28:26.355165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.845762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:29.175175+0000","last_change":"2024-04-25T00:28:23.410964+0000","last_active":"2024-04-25T00:28:29.175175+0000","last_peered":"2024-04-25T00:28:29.175175+0000","last_clean":"2024-04-25T00:28:29.175175+0000","last_became_active":"2024-04-25T00:28:23.410602+0000","last_became_peered":"2024-04-25T00:28:23.410602+0000","last_unstale":"2024-04-25T00:28:29.175175+0000","last_undegraded":"2024-04-25T00:28:29.175175+0000","last_fullsized":"2024-04-25T00:28:29.175175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:24.495790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355802+0000","last_change":"2024-04-25T00:28:24.184710+0000","last_active":"2024-04-25T00:28:26.355802+0000","last_peered":"2024-04-25T00:28:26.355802+0000","last_clean":"2024-04-25T00:28:26.355802+0000","last_became_active":"2024-04-25T00:28:24.184354+0000","last_became_peered":"2024-04-25T00:28:24.184354+0000","last_unstale":"2024-04-25T00:28:26.355802+0000","last_undegraded":"2024-04-25T00:28:26.355802+0000","last_fullsized":"2024-04-25T00:28:26.355802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:47.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354533+0000","last_change":"2024-04-25T00:28:24.180268+0000","last_active":"2024-04-25T00:28:26.354533+0000","last_peered":"2024-04-25T00:28:26.354533+0000","last_clean":"2024-04-25T00:28:26.354533+0000","last_became_active":"2024-04-25T00:28:24.178941+0000","last_became_peered":"2024-04-25T00:28:24.178941+0000","last_unstale":"2024-04-25T00:28:26.354533+0000","last_undegraded":"2024-04-25T00:28:26.354533+0000","last_fullsized":"2024-04-25T00:28:26.354533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:06.861257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.444421+0000","last_change":"2024-04-25T00:28:23.401437+0000","last_active":"2024-04-25T00:28:27.444421+0000","last_peered":"2024-04-25T00:28:27.444421+0000","last_clean":"2024-04-25T00:28:27.444421+0000","last_became_active":"2024-04-25T00:28:23.401091+0000","last_became_peered":"2024-04-25T00:28:23.401091+0000","last_unstale":"2024-04-25T00:28:27.444421+0000","last_undegraded":"2024-04-25T00:28:27.444421+0000","last_fullsized":"2024-04-25T00:28:27.444421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:58.748461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414209+0000","last_change":"2024-04-25T00:28:24.166224+0000","last_active":"2024-04-25T00:28:25.414209+0000","last_peered":"2024-04-25T00:28:25.414209+0000","last_clean":"2024-04-25T00:28:25.414209+0000","last_became_active":"2024-04-25T00:28:24.165626+0000","last_became_peered":"2024-04-25T00:28:24.165626+0000","last_unstale":"2024-04-25T00:28:25.414209+0000","last_undegraded":"2024-04-25T00:28:25.414209+0000","last_fullsized":"2024-04-25T00:28:25.414209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:04.956168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435448+0000","last_change":"2024-04-25T00:28:23.408551+0000","last_active":"2024-04-25T00:28:26.435448+0000","last_peered":"2024-04-25T00:28:26.435448+0000","last_clean":"2024-04-25T00:28:26.435448+0000","last_became_active":"2024-04-25T00:28:23.408137+0000","last_became_peered":"2024-04-25T00:28:23.408137+0000","last_unstale":"2024-04-25T00:28:26.435448+0000","last_undegraded":"2024-04-25T00:28:26.435448+0000","last_fullsized":"2024-04-25T00:28:26.435448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:53.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354408+0000","last_change":"2024-04-25T00:27:10.587192+0000","last_active":"2024-04-25T00:28:26.354408+0000","last_peered":"2024-04-25T00:28:26.354408+0000","last_clean":"2024-04-25T00:28:26.354408+0000","last_became_active":"2024-04-25T00:27:10.586481+0000","last_became_peered":"2024-04-25T00:27:10.586481+0000","last_unstale":"2024-04-25T00:28:26.354408+0000","last_undegraded":"2024-04-25T00:28:26.354408+0000","last_fullsized":"2024-04-25T00:28:26.354408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_clean_scrub_stamp":"2024-04-25T00:23:42.305887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:44.337443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354449+0000","last_change":"2024-04-25T00:28:24.183078+0000","last_active":"2024-04-25T00:28:26.354449+0000","last_peered":"2024-04-25T00:28:26.354449+0000","last_clean":"2024-04-25T00:28:26.354449+0000","last_became_active":"2024-04-25T00:28:24.182657+0000","last_became_peered":"2024-04-25T00:28:24.182657+0000","last_unstale":"2024-04-25T00:28:26.354449+0000","last_undegraded":"2024-04-25T00:28:26.354449+0000","last_fullsized":"2024-04-25T00:28:26.354449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:14.989908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:27.443170+0000","last_change":"2024-04-25T00:28:23.404002+0000","last_active":"2024-04-25T00:28:27.443170+0000","last_peered":"2024-04-25T00:28:27.443170+0000","last_clean":"2024-04-25T00:28:27.443170+0000","last_became_active":"2024-04-25T00:28:23.403659+0000","last_became_peered":"2024-04-25T00:28:23.403659+0000","last_unstale":"2024-04-25T00:28:27.443170+0000","last_undegraded":"2024-04-25T00:28:27.443170+0000","last_fullsized":"2024-04-25T00:28:27.443170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:26.994242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161551+0000","last_change":"2024-04-25T00:28:23.405525+0000","last_active":"2024-04-25T00:28:25.161551+0000","last_peered":"2024-04-25T00:28:25.161551+0000","last_clean":"2024-04-25T00:28:25.161551+0000","last_became_active":"2024-04-25T00:28:23.405228+0000","last_became_peered":"2024-04-25T00:28:23.405228+0000","last_unstale":"2024-04-25T00:28:25.161551+0000","last_undegraded":"2024-04-25T00:28:25.161551+0000","last_fullsized":"2024-04-25T00:28:25.161551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:00.819210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354771+0000","last_change":"2024-04-25T00:28:24.182350+0000","last_active":"2024-04-25T00:28:26.354771+0000","last_peered":"2024-04-25T00:28:26.354771+0000","last_clean":"2024-04-25T00:28:26.354771+0000","last_became_active":"2024-04-25T00:28:24.182002+0000","last_became_peered":"2024-04-25T00:28:24.182002+0000","last_unstale":"2024-04-25T00:28:26.354771+0000","last_undegraded":"2024-04-25T00:28:26.354771+0000","last_fullsized":"2024-04-25T00:28:26.354771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:55.891046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.411359+0000","last_change":"2024-04-25T00:28:23.405357+0000","last_active":"2024-04-25T00:28:25.411359+0000","last_peered":"2024-04-25T00:28:25.411359+0000","last_clean":"2024-04-25T00:28:25.411359+0000","last_became_active":"2024-04-25T00:28:23.405031+0000","last_became_peered":"2024-04-25T00:28:23.405031+0000","last_unstale":"2024-04-25T00:28:25.411359+0000","last_undegraded":"2024-04-25T00:28:25.411359+0000","last_fullsized":"2024-04-25T00:28:25.411359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:10.683462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414385+0000","last_change":"2024-04-25T00:28:24.177859+0000","last_active":"2024-04-25T00:28:25.414385+0000","last_peered":"2024-04-25T00:28:25.414385+0000","last_clean":"2024-04-25T00:28:25.414385+0000","last_became_active":"2024-04-25T00:28:24.177533+0000","last_became_peered":"2024-04-25T00:28:24.177533+0000","last_unstale":"2024-04-25T00:28:25.414385+0000","last_undegraded":"2024-04-25T00:28:25.414385+0000","last_fullsized":"2024-04-25T00:28:25.414385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:46.964934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.433785+0000","last_change":"2024-04-25T00:28:23.409487+0000","last_active":"2024-04-25T00:28:26.433785+0000","last_peered":"2024-04-25T00:28:26.433785+0000","last_clean":"2024-04-25T00:28:26.433785+0000","last_became_active":"2024-04-25T00:28:23.409146+0000","last_became_peered":"2024-04-25T00:28:23.409146+0000","last_unstale":"2024-04-25T00:28:26.433785+0000","last_undegraded":"2024-04-25T00:28:26.433785+0000","last_fullsized":"2024-04-25T00:28:26.433785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.591409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160340+0000","last_change":"2024-04-25T00:28:24.185083+0000","last_active":"2024-04-25T00:28:25.160340+0000","last_peered":"2024-04-25T00:28:25.160340+0000","last_clean":"2024-04-25T00:28:25.160340+0000","last_became_active":"2024-04-25T00:28:24.184752+0000","last_became_peered":"2024-04-25T00:28:24.184752+0000","last_unstale":"2024-04-25T00:28:25.160340+0000","last_undegraded":"2024-04-25T00:28:25.160340+0000","last_fullsized":"2024-04-25T00:28:25.160340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:08.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.434198+0000","last_change":"2024-04-25T00:28:23.405662+0000","last_active":"2024-04-25T00:28:26.434198+0000","last_peered":"2024-04-25T00:28:26.434198+0000","last_clean":"2024-04-25T00:28:26.434198+0000","last_became_active":"2024-04-25T00:28:23.404949+0000","last_became_peered":"2024-04-25T00:28:23.404949+0000","last_unstale":"2024-04-25T00:28:26.434198+0000","last_undegraded":"2024-04-25T00:28:26.434198+0000","last_fullsized":"2024-04-25T00:28:26.434198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:01.989041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.161234+0000","last_change":"2024-04-25T00:28:24.181404+0000","last_active":"2024-04-25T00:28:25.161234+0000","last_peered":"2024-04-25T00:28:25.161234+0000","last_clean":"2024-04-25T00:28:25.161234+0000","last_became_active":"2024-04-25T00:28:24.180399+0000","last_became_peered":"2024-04-25T00:28:24.180399+0000","last_unstale":"2024-04-25T00:28:25.161234+0000","last_undegraded":"2024-04-25T00:28:25.161234+0000","last_fullsized":"2024-04-25T00:28:25.161234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:41.955204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.434284+0000","last_change":"2024-04-25T00:28:23.405288+0000","last_active":"2024-04-25T00:28:26.434284+0000","last_peered":"2024-04-25T00:28:26.434284+0000","last_clean":"2024-04-25T00:28:26.434284+0000","last_became_active":"2024-04-25T00:28:23.404547+0000","last_became_peered":"2024-04-25T00:28:23.404547+0000","last_unstale":"2024-04-25T00:28:26.434284+0000","last_undegraded":"2024-04-25T00:28:26.434284+0000","last_fullsized":"2024-04-25T00:28:26.434284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:45.180422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392336+0000","last_change":"2024-04-25T00:28:24.176840+0000","last_active":"2024-04-25T00:28:25.392336+0000","last_peered":"2024-04-25T00:28:25.392336+0000","last_clean":"2024-04-25T00:28:25.392336+0000","last_became_active":"2024-04-25T00:28:24.176478+0000","last_became_peered":"2024-04-25T00:28:24.176478+0000","last_unstale":"2024-04-25T00:28:25.392336+0000","last_undegraded":"2024-04-25T00:28:25.392336+0000","last_fullsized":"2024-04-25T00:28:25.392336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:50.057929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.168809+0000","last_change":"2024-04-25T00:28:23.400594+0000","last_active":"2024-04-25T00:28:24.168809+0000","last_peered":"2024-04-25T00:28:24.168809+0000","last_clean":"2024-04-25T00:28:24.168809+0000","last_became_active":"2024-04-25T00:28:23.400216+0000","last_became_peered":"2024-04-25T00:28:23.400216+0000","last_unstale":"2024-04-25T00:28:24.168809+0000","last_undegraded":"2024-04-25T00:28:24.168809+0000","last_fullsized":"2024-04-25T00:28:24.168809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:18.771989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414191+0000","last_change":"2024-04-25T00:28:24.176402+0000","last_active":"2024-04-25T00:28:25.414191+0000","last_peered":"2024-04-25T00:28:25.414191+0000","last_clean":"2024-04-25T00:28:25.414191+0000","last_became_active":"2024-04-25T00:28:24.175822+0000","last_became_peered":"2024-04-25T00:28:24.175822+0000","last_unstale":"2024-04-25T00:28:25.414191+0000","last_undegraded":"2024-04-25T00:28:25.414191+0000","last_fullsized":"2024-04-25T00:28:25.414191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.952116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.435131+0000","last_change":"2024-04-25T00:28:23.401158+0000","last_active":"2024-04-25T00:28:26.435131+0000","last_peered":"2024-04-25T00:28:26.435131+0000","last_clean":"2024-04-25T00:28:26.435131+0000","last_became_active":"2024-04-25T00:28:23.400819+0000","last_became_peered":"2024-04-25T00:28:23.400819+0000","last_unstale":"2024-04-25T00:28:26.435131+0000","last_undegraded":"2024-04-25T00:28:26.435131+0000","last_fullsized":"2024-04-25T00:28:26.435131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.209636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355537+0000","last_change":"2024-04-25T00:28:24.189784+0000","last_active":"2024-04-25T00:28:26.355537+0000","last_peered":"2024-04-25T00:28:26.355537+0000","last_clean":"2024-04-25T00:28:26.355537+0000","last_became_active":"2024-04-25T00:28:24.188161+0000","last_became_peered":"2024-04-25T00:28:24.188161+0000","last_unstale":"2024-04-25T00:28:26.355537+0000","last_undegraded":"2024-04-25T00:28:26.355537+0000","last_fullsized":"2024-04-25T00:28:26.355537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:06.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.165763+0000","last_change":"2024-04-25T00:28:23.410684+0000","last_active":"2024-04-25T00:28:24.165763+0000","last_peered":"2024-04-25T00:28:24.165763+0000","last_clean":"2024-04-25T00:28:24.165763+0000","last_became_active":"2024-04-25T00:28:23.410345+0000","last_became_peered":"2024-04-25T00:28:23.410345+0000","last_unstale":"2024-04-25T00:28:24.165763+0000","last_undegraded":"2024-04-25T00:28:24.165763+0000","last_fullsized":"2024-04-25T00:28:24.165763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355831+0000","last_change":"2024-04-25T00:28:24.189170+0000","last_active":"2024-04-25T00:28:26.355831+0000","last_peered":"2024-04-25T00:28:26.355831+0000","last_clean":"2024-04-25T00:28:26.355831+0000","last_became_active":"2024-04-25T00:28:24.188329+0000","last_became_peered":"2024-04-25T00:28:24.188329+0000","last_unstale":"2024-04-25T00:28:26.355831+0000","last_undegraded":"2024-04-25T00:28:26.355831+0000","last_fullsized":"2024-04-25T00:28:26.355831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:41.523820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354766+0000","last_change":"2024-04-25T00:28:23.405933+0000","last_active":"2024-04-25T00:28:26.354766+0000","last_peered":"2024-04-25T00:28:26.354766+0000","last_clean":"2024-04-25T00:28:26.354766+0000","last_became_active":"2024-04-25T00:28:23.405585+0000","last_became_peered":"2024-04-25T00:28:23.405585+0000","last_unstale":"2024-04-25T00:28:26.354766+0000","last_undegraded":"2024-04-25T00:28:26.354766+0000","last_fullsized":"2024-04-25T00:28:26.354766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:05.914735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354782+0000","last_change":"2024-04-25T00:28:24.178720+0000","last_active":"2024-04-25T00:28:26.354782+0000","last_peered":"2024-04-25T00:28:26.354782+0000","last_clean":"2024-04-25T00:28:26.354782+0000","last_became_active":"2024-04-25T00:28:24.178270+0000","last_became_peered":"2024-04-25T00:28:24.178270+0000","last_unstale":"2024-04-25T00:28:26.354782+0000","last_undegraded":"2024-04-25T00:28:26.354782+0000","last_fullsized":"2024-04-25T00:28:26.354782+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:38.076554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354736+0000","last_change":"2024-04-25T00:28:23.403017+0000","last_active":"2024-04-25T00:28:26.354736+0000","last_peered":"2024-04-25T00:28:26.354736+0000","last_clean":"2024-04-25T00:28:26.354736+0000","last_became_active":"2024-04-25T00:28:23.402706+0000","last_became_peered":"2024-04-25T00:28:23.402706+0000","last_unstale":"2024-04-25T00:28:26.354736+0000","last_undegraded":"2024-04-25T00:28:26.354736+0000","last_fullsized":"2024-04-25T00:28:26.354736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:06.916509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.355042+0000","last_change":"2024-04-25T00:28:24.181585+0000","last_active":"2024-04-25T00:28:26.355042+0000","last_peered":"2024-04-25T00:28:26.355042+0000","last_clean":"2024-04-25T00:28:26.355042+0000","last_became_active":"2024-04-25T00:28:24.180755+0000","last_became_peered":"2024-04-25T00:28:24.180755+0000","last_unstale":"2024-04-25T00:28:26.355042+0000","last_undegraded":"2024-04-25T00:28:26.355042+0000","last_fullsized":"2024-04-25T00:28:26.355042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:59.711999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392418+0000","last_change":"2024-04-25T00:28:24.175444+0000","last_active":"2024-04-25T00:28:25.392418+0000","last_peered":"2024-04-25T00:28:25.392418+0000","last_clean":"2024-04-25T00:28:25.392418+0000","last_became_active":"2024-04-25T00:28:24.174322+0000","last_became_peered":"2024-04-25T00:28:24.174322+0000","last_unstale":"2024-04-25T00:28:25.392418+0000","last_undegraded":"2024-04-25T00:28:25.392418+0000","last_fullsized":"2024-04-25T00:28:25.392418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:15.325122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.189100+0000","last_change":"2024-04-25T00:28:24.188603+0000","last_active":"2024-04-25T00:28:24.189100+0000","last_peered":"2024-04-25T00:28:24.189100+0000","last_clean":"2024-04-25T00:28:24.189100+0000","last_became_active":"2024-04-25T00:28:24.188300+0000","last_became_peered":"2024-04-25T00:28:24.188300+0000","last_unstale":"2024-04-25T00:28:24.189100+0000","last_undegraded":"2024-04-25T00:28:24.189100+0000","last_fullsized":"2024-04-25T00:28:24.189100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:39.221786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.184900+0000","last_change":"2024-04-25T00:28:24.183907+0000","last_active":"2024-04-25T00:28:24.184900+0000","last_peered":"2024-04-25T00:28:24.184900+0000","last_clean":"2024-04-25T00:28:24.184900+0000","last_became_active":"2024-04-25T00:28:24.183545+0000","last_became_peered":"2024-04-25T00:28:24.183545+0000","last_unstale":"2024-04-25T00:28:24.184900+0000","last_undegraded":"2024-04-25T00:28:24.184900+0000","last_fullsized":"2024-04-25T00:28:24.184900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:07.606815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.160542+0000","last_change":"2024-04-25T00:28:24.178516+0000","last_active":"2024-04-25T00:28:25.160542+0000","last_peered":"2024-04-25T00:28:25.160542+0000","last_clean":"2024-04-25T00:28:25.160542+0000","last_became_active":"2024-04-25T00:28:24.177995+0000","last_became_peered":"2024-04-25T00:28:24.177995+0000","last_unstale":"2024-04-25T00:28:25.160542+0000","last_undegraded":"2024-04-25T00:28:25.160542+0000","last_fullsized":"2024-04-25T00:28:25.160542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:03.396880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354817+0000","last_change":"2024-04-25T00:28:24.175401+0000","last_active":"2024-04-25T00:28:26.354817+0000","last_peered":"2024-04-25T00:28:26.354817+0000","last_clean":"2024-04-25T00:28:26.354817+0000","last_became_active":"2024-04-25T00:28:24.174648+0000","last_became_peered":"2024-04-25T00:28:24.174648+0000","last_unstale":"2024-04-25T00:28:26.354817+0000","last_undegraded":"2024-04-25T00:28:26.354817+0000","last_fullsized":"2024-04-25T00:28:26.354817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:00.630810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392578+0000","last_change":"2024-04-25T00:28:24.173542+0000","last_active":"2024-04-25T00:28:25.392578+0000","last_peered":"2024-04-25T00:28:25.392578+0000","last_clean":"2024-04-25T00:28:25.392578+0000","last_became_active":"2024-04-25T00:28:24.173240+0000","last_became_peered":"2024-04-25T00:28:24.173240+0000","last_unstale":"2024-04-25T00:28:25.392578+0000","last_undegraded":"2024-04-25T00:28:25.392578+0000","last_fullsized":"2024-04-25T00:28:25.392578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:00.189612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:28:24.186857+0000","last_change":"2024-04-25T00:28:24.185104+0000","last_active":"2024-04-25T00:28:24.186857+0000","last_peered":"2024-04-25T00:28:24.186857+0000","last_clean":"2024-04-25T00:28:24.186857+0000","last_became_active":"2024-04-25T00:28:24.184566+0000","last_became_peered":"2024-04-25T00:28:24.184566+0000","last_unstale":"2024-04-25T00:28:24.186857+0000","last_undegraded":"2024-04-25T00:28:24.186857+0000","last_fullsized":"2024-04-25T00:28:24.186857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:13.287970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.414302+0000","last_change":"2024-04-25T00:28:24.181286+0000","last_active":"2024-04-25T00:28:25.414302+0000","last_peered":"2024-04-25T00:28:25.414302+0000","last_clean":"2024-04-25T00:28:25.414302+0000","last_became_active":"2024-04-25T00:28:24.180406+0000","last_became_peered":"2024-04-25T00:28:24.180406+0000","last_unstale":"2024-04-25T00:28:25.414302+0000","last_undegraded":"2024-04-25T00:28:25.414302+0000","last_fullsized":"2024-04-25T00:28:25.414302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.644228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354437+0000","last_change":"2024-04-25T00:28:24.169883+0000","last_active":"2024-04-25T00:28:26.354437+0000","last_peered":"2024-04-25T00:28:26.354437+0000","last_clean":"2024-04-25T00:28:26.354437+0000","last_became_active":"2024-04-25T00:28:24.169558+0000","last_became_peered":"2024-04-25T00:28:24.169558+0000","last_unstale":"2024-04-25T00:28:26.354437+0000","last_undegraded":"2024-04-25T00:28:26.354437+0000","last_fullsized":"2024-04-25T00:28:26.354437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:35.615978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.413793+0000","last_change":"2024-04-25T00:28:24.180521+0000","last_active":"2024-04-25T00:28:25.413793+0000","last_peered":"2024-04-25T00:28:25.413793+0000","last_clean":"2024-04-25T00:28:25.413793+0000","last_became_active":"2024-04-25T00:28:24.179867+0000","last_became_peered":"2024-04-25T00:28:24.179867+0000","last_unstale":"2024-04-25T00:28:25.413793+0000","last_undegraded":"2024-04-25T00:28:25.413793+0000","last_fullsized":"2024-04-25T00:28:25.413793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:31.660585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354530+0000","last_change":"2024-04-25T00:28:24.185050+0000","last_active":"2024-04-25T00:28:26.354530+0000","last_peered":"2024-04-25T00:28:26.354530+0000","last_clean":"2024-04-25T00:28:26.354530+0000","last_became_active":"2024-04-25T00:28:24.184663+0000","last_became_peered":"2024-04-25T00:28:24.184663+0000","last_unstale":"2024-04-25T00:28:26.354530+0000","last_undegraded":"2024-04-25T00:28:26.354530+0000","last_fullsized":"2024-04-25T00:28:26.354530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:32.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:26.354677+0000","last_change":"2024-04-25T00:28:24.177948+0000","last_active":"2024-04-25T00:28:26.354677+0000","last_peered":"2024-04-25T00:28:26.354677+0000","last_clean":"2024-04-25T00:28:26.354677+0000","last_became_active":"2024-04-25T00:28:24.177627+0000","last_became_peered":"2024-04-25T00:28:24.177627+0000","last_unstale":"2024-04-25T00:28:26.354677+0000","last_undegraded":"2024-04-25T00:28:26.354677+0000","last_fullsized":"2024-04-25T00:28:26.354677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:29.128868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:28:25.392492+0000","last_change":"2024-04-25T00:28:24.176115+0000","last_active":"2024-04-25T00:28:25.392492+0000","last_peered":"2024-04-25T00:28:25.392492+0000","last_clean":"2024-04-25T00:28:25.392492+0000","last_became_active":"2024-04-25T00:28:24.175735+0000","last_became_peered":"2024-04-25T00:28:24.175735+0000","last_unstale":"2024-04-25T00:28:25.392492+0000","last_undegraded":"2024-04-25T00:28:25.392492+0000","last_fullsized":"2024-04-25T00:28:25.392492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:58.595542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280907,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5228,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":111,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":16599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":447512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":1039866,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":447714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266723,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1040831,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":447020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":448542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":447624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":440006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":447034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":447574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1028910,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:28:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:28:31.330 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T00:28:31.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T00:28:31.611 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] up:active 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4071190719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:31.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:31 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/162358812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] up:active 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4071190719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:31 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/162358812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:32.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:32 smithi031 ceph-mon[29167]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:28:32.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:32 smithi031 ceph-mon[29167]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 129 B/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-25T00:28:32.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:32 smithi031 ceph-mon[29167]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:28:32.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:32 smithi169 ceph-mon[33112]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:28:32.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:32 smithi169 ceph-mon[33112]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 129 B/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-25T00:28:32.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:32 smithi169 ceph-mon[33112]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:28:32.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:32 smithi040 ceph-mon[32945]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:28:32.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:32 smithi040 ceph-mon[32945]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 129 B/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-25T00:28:32.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:32 smithi040 ceph-mon[32945]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:28:33.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.433+0000 7fda11477700 1 -- 172.21.15.31:0/1189231161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c107fc0 msgr2=0x7fda0c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.433+0000 7fda11477700 1 --2- 172.21.15.31:0/1189231161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c107fc0 0x7fda0c10a430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda0400b0a0 tx=0x7fda0400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 -- 172.21.15.31:0/1189231161 shutdown_connections 2024-04-25T00:28:33.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 --2- 172.21.15.31:0/1189231161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c107fc0 0x7fda0c10a430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 --2- 172.21.15.31:0/1189231161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c102d60 0x7fda0c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 --2- 172.21.15.31:0/1189231161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105690 0x7fda0c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 -- 172.21.15.31:0/1189231161 >> 172.21.15.31:0/1189231161 conn(0x7fda0c0fc790 msgr2=0x7fda0c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 -- 172.21.15.31:0/1189231161 shutdown_connections 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 -- 172.21.15.31:0/1189231161 wait complete. 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 Processor -- start 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.434+0000 7fda11477700 1 -- start start 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 0x7fda0c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105690 0x7fda0c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c107fc0 0x7fda0c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10dd30 con 0x7fda0c102d60 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10dbb0 con 0x7fda0c107fc0 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10deb0 con 0x7fda0c105690 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 0x7fda0c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 0x7fda0c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46968/0 (socket says 172.21.15.31:46968) 2024-04-25T00:28:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 -- 172.21.15.31:0/2277551990 learned_addr learned my addr 172.21.15.31:0/2277551990 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0a7fc700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105690 0x7fda0c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 -- 172.21.15.31:0/2277551990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105690 msgr2=0x7fda0c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105690 0x7fda0c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0b7fe700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c107fc0 0x7fda0c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 -- 172.21.15.31:0/2277551990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c107fc0 msgr2=0x7fda0c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c107fc0 0x7fda0c19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 -- 172.21.15.31:0/2277551990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c19fe80 con 0x7fda0c102d60 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda0affd700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 0x7fda0c198220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c9e0 tx=0x7fd9fc00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc007ac0 con 0x7fda0c102d60 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9fc007c60 con 0x7fda0c102d60 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1a0170 con 0x7fda0c102d60 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.435+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc018660 con 0x7fda0c102d60 2024-04-25T00:28:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.436+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a06b0 con 0x7fda0c102d60 2024-04-25T00:28:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.436+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c0507a0 con 0x7fda0c102d60 2024-04-25T00:28:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.438+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd9fc00f450 con 0x7fda0c102d60 2024-04-25T00:28:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.438+0000 7fd9f3fff700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd9f40a2d60 0x7fd9f40a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.439+0000 7fda0a7fc700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd9f40a2d60 0x7fd9f40a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.439+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd9fc0cf000 con 0x7fda0c102d60 2024-04-25T00:28:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.439+0000 7fda0a7fc700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd9f40a2d60 0x7fd9f40a5220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda00005fd0 tx=0x7fda00005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.443+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd9fc014030 con 0x7fda0c102d60 2024-04-25T00:28:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.737+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fda0c02dbd0 con 0x7fda0c102d60 2024-04-25T00:28:33.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.739+0000 7fd9f3fff700 1 -- 172.21.15.31:0/2277551990 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd9fc09bfb0 con 0x7fda0c102d60 2024-04-25T00:28:33.740 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-04-25T00:28:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCxoylmMJQKLBAAzs/U2ZamQLO+P5MeBg/2CQ== 2024-04-25T00:28:33.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd9f40a2d60 msgr2=0x7fd9f40a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd9f40a2d60 0x7fd9f40a5220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda00005fd0 tx=0x7fda00005f60 comp rx=0 tx=0).stop 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 msgr2=0x7fda0c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 0x7fda0c198220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c9e0 tx=0x7fd9fc00cda0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 shutdown_connections 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd9f40a2d60 0x7fd9f40a5220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c102d60 0x7fda0c198220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c107fc0 0x7fda0c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 --2- 172.21.15.31:0/2277551990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105690 0x7fda0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 >> 172.21.15.31:0/2277551990 conn(0x7fda0c0fc790 msgr2=0x7fda0c1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 shutdown_connections 2024-04-25T00:28:33.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:33.742+0000 7fda11477700 1 -- 172.21.15.31:0/2277551990 wait complete. 2024-04-25T00:28:34.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:34 smithi031 ceph-mon[29167]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 31 op/s 2024-04-25T00:28:34.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2277551990' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:28:34.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2277551990' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:28:34.536 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:28:34.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T00:28:34.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:28:34.574 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:28:34.574 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:28:34.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T00:28:34.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:34 smithi169 ceph-mon[33112]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 31 op/s 2024-04-25T00:28:34.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2277551990' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:28:34.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2277551990' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:28:34.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:34 smithi040 ceph-mon[32945]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 31 op/s 2024-04-25T00:28:34.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2277551990' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:28:34.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2277551990' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:28:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:36.617 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:36 smithi031 ceph-mon[29167]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 29 op/s 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 -- 172.21.15.31:0/4220815361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102ec0 msgr2=0x7f94f0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 --2- 172.21.15.31:0/4220815361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102ec0 0x7f94f0106f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94d80099c0 tx=0x7f94d8009cd0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 -- 172.21.15.31:0/4220815361 shutdown_connections 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 --2- 172.21.15.31:0/4220815361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f01075e0 0x7f94f010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 --2- 172.21.15.31:0/4220815361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0102510 0x7f94f01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 --2- 172.21.15.31:0/4220815361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102ec0 0x7f94f0106f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 -- 172.21.15.31:0/4220815361 >> 172.21.15.31:0/4220815361 conn(0x7f94f00793e0 msgr2=0x7f94f00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 -- 172.21.15.31:0/4220815361 shutdown_connections 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.616+0000 7f94f56f5700 1 -- 172.21.15.31:0/4220815361 wait complete. 2024-04-25T00:28:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 Processor -- start 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 -- start start 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 0x7f94f0074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0102ec0 0x7f94f0074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f01075e0 0x7f94f0074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94f01124d0 con 0x7f94f0102510 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94f0112350 con 0x7f94f0102ec0 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94f56f5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94f0112650 con 0x7f94f01075e0 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 0x7f94f0074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 0x7f94f0074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46996/0 (socket says 172.21.15.31:46996) 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94eeffd700 1 -- 172.21.15.31:0/1089492587 learned_addr learned my addr 172.21.15.31:0/1089492587 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.617+0000 7f94ee7fc700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0102ec0 0x7f94f0074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94ef7fe700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f01075e0 0x7f94f0074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94eeffd700 1 -- 172.21.15.31:0/1089492587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f01075e0 msgr2=0x7f94f0074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94eeffd700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f01075e0 0x7f94f0074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94eeffd700 1 -- 172.21.15.31:0/1089492587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0102ec0 msgr2=0x7f94f0074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94eeffd700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0102ec0 0x7f94f0074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94eeffd700 1 -- 172.21.15.31:0/1089492587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f01a6f50 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94eeffd700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 0x7f94f0074530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94e80060b0 tx=0x7f94e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e801a070 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f01a7240 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94e800ddd0 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8014720 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.618+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f01a77a0 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.619+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f01039e0 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.620+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f94e801d070 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.621+0000 7f94e7fff700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f94dc0a2cb0 0x7f94dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.621+0000 7f94ee7fc700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f94dc0a2cb0 0x7f94dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.621+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f94e80ce8c0 con 0x7f94f0102510 2024-04-25T00:28:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.621+0000 7f94ee7fc700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f94dc0a2cb0 0x7f94dc0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f94d8000c00 tx=0x7f94d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.625+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f94e8010070 con 0x7f94f0102510 2024-04-25T00:28:36.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:36 smithi169 ceph-mon[33112]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 29 op/s 2024-04-25T00:28:36.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:36 smithi040 ceph-mon[32945]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 29 op/s 2024-04-25T00:28:36.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.891+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f94f0075a60 con 0x7f94f0102510 2024-04-25T00:28:36.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.896+0000 7f94e7fff700 1 -- 172.21.15.31:0/1089492587 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7f94e809b870 con 0x7f94f0102510 2024-04-25T00:28:36.900 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f94dc0a2cb0 msgr2=0x7f94dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f94dc0a2cb0 0x7f94dc0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f94d8000c00 tx=0x7f94d8005fb0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 msgr2=0x7f94f0074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 0x7f94f0074530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94e80060b0 tx=0x7f94e800d3b0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 shutdown_connections 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f94dc0a2cb0 0x7f94dc0a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f0102510 0x7f94f0074530 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f01075e0 0x7f94f0074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 --2- 172.21.15.31:0/1089492587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0102ec0 0x7f94f0074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 >> 172.21.15.31:0/1089492587 conn(0x7f94f00793e0 msgr2=0x7f94f0100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 shutdown_connections 2024-04-25T00:28:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:36.900+0000 7f94f56f5700 1 -- 172.21.15.31:0/1089492587 wait complete. 2024-04-25T00:28:37.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:37 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1089492587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:28:37.640 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:37 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1089492587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:28:37.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:37 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1089492587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:28:37.672 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2153348461},{"type":"v1","addr":"172.21.15.31:6801","nonce":2153348461}]},"active_addr":"172.21.15.31:6801/2153348461","active_change":"2024-04-25T00:21:13.218846+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24118,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":128753718}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3409453448}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":4147807999}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":209666436}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":438565003}]}]} 2024-04-25T00:28:37.678 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T00:28:37.679 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T00:28:37.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:28:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:38.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:38 smithi169 ceph-mon[33112]: pgmap v299: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-04-25T00:28:38.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:38 smithi040 ceph-mon[32945]: pgmap v299: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-04-25T00:28:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:38 smithi031 ceph-mon[29167]: pgmap v299: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-04-25T00:28:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 -- 172.21.15.31:0/777413019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3481082c0 msgr2=0x7ff3481086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 --2- 172.21.15.31:0/777413019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3481082c0 0x7ff3481086a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b0a0 tx=0x7ff34400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 -- 172.21.15.31:0/777413019 shutdown_connections 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 --2- 172.21.15.31:0/777413019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3481082c0 0x7ff3481086a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 --2- 172.21.15.31:0/777413019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 0x7ff34810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 --2- 172.21.15.31:0/777413019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff348100820 0x7ff348100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 -- 172.21.15.31:0/777413019 >> 172.21.15.31:0/777413019 conn(0x7ff3480777a0 msgr2=0x7ff3480feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 -- 172.21.15.31:0/777413019 shutdown_connections 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.773+0000 7ff34f975700 1 -- 172.21.15.31:0/777413019 wait complete. 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 Processor -- start 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 -- start start 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 0x7ff348101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 0x7ff348101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3481082c0 0x7ff348102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481123b0 con 0x7ff348100820 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff348112230 con 0x7ff3481082c0 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34f975700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff348112530 con 0x7ff3481011c0 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34d711700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 0x7ff348101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34d711700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 0x7ff348101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47014/0 (socket says 172.21.15.31:47014) 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.774+0000 7ff34d711700 1 -- 172.21.15.31:0/2553138680 learned_addr learned my addr 172.21.15.31:0/2553138680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34df12700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3481082c0 0x7ff348102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34cf10700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 0x7ff348101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34d711700 1 -- 172.21.15.31:0/2553138680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 msgr2=0x7ff348101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34d711700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 0x7ff348101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34d711700 1 -- 172.21.15.31:0/2553138680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3481082c0 msgr2=0x7ff348102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34d711700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3481082c0 0x7ff348102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34d711700 1 -- 172.21.15.31:0/2553138680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3481a6d90 con 0x7ff348100820 2024-04-25T00:28:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34cf10700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 0x7ff348101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34d711700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 0x7ff348101a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b070 tx=0x7ff344004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344021070 con 0x7ff348100820 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff344007e30 con 0x7ff348100820 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3481a7020 con 0x7ff348100820 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004840 con 0x7ff348100820 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.775+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a74d0 con 0x7ff348100820 2024-04-25T00:28:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.776+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff348103b40 con 0x7ff348100820 2024-04-25T00:28:39.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.778+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff344003c00 con 0x7ff348100820 2024-04-25T00:28:39.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.779+0000 7ff33e7fc700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3340a2cb0 0x7ff3340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:39.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.779+0000 7ff34cf10700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3340a2cb0 0x7ff3340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:39.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.779+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff34400f020 con 0x7ff348100820 2024-04-25T00:28:39.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.779+0000 7ff34cf10700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3340a2cb0 0x7ff3340a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff3380058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:39.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.783+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff34402a020 con 0x7ff348100820 2024-04-25T00:28:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.989+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff34802db30 con 0x7ff348100820 2024-04-25T00:28:39.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.990+0000 7ff33e7fc700 1 -- 172.21.15.31:0/2553138680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22209 (secure 0 0 0) 0x7ff34409d930 con 0x7ff348100820 2024-04-25T00:28:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:39.991 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:28:25.157023+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:28:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3340a2cb0 msgr2=0x7ff3340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3340a2cb0 0x7ff3340a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff3380058e0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 msgr2=0x7ff348101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 0x7ff348101a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b070 tx=0x7ff344004d00 comp rx=0 tx=0).stop 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 shutdown_connections 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3340a2cb0 0x7ff3340a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff348100820 0x7ff348101a00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3481082c0 0x7ff348102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 --2- 172.21.15.31:0/2553138680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3481011c0 0x7ff348101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 >> 172.21.15.31:0/2553138680 conn(0x7ff3480777a0 msgr2=0x7ff3480fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 shutdown_connections 2024-04-25T00:28:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:39.994+0000 7ff34f975700 1 -- 172.21.15.31:0/2553138680 wait complete. 2024-04-25T00:28:40.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:40 smithi169 ceph-mon[33112]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 28 op/s 2024-04-25T00:28:40.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:40 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2553138680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:40.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:40 smithi040 ceph-mon[32945]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 28 op/s 2024-04-25T00:28:40.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:40 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2553138680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:40 smithi031 ceph-mon[29167]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 28 op/s 2024-04-25T00:28:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:40 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2553138680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:40.817 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T00:28:40.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:28:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:42.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:42 smithi031 ceph-mon[29167]: pgmap v301: 97 pgs: 97 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-04-25T00:28:42.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:42 smithi169 ceph-mon[33112]: pgmap v301: 97 pgs: 97 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-04-25T00:28:42.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:42 smithi040 ceph-mon[32945]: pgmap v301: 97 pgs: 97 active+clean; 630 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.814+0000 7fd96769c700 1 -- 172.21.15.31:0/3187540461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 msgr2=0x7fd96010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.814+0000 7fd96769c700 1 --2- 172.21.15.31:0/3187540461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd96010d770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd95800b0a0 tx=0x7fd95800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- 172.21.15.31:0/3187540461 shutdown_connections 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 --2- 172.21.15.31:0/3187540461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd96010d770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 --2- 172.21.15.31:0/3187540461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9601068a0 0x7fd960106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 --2- 172.21.15.31:0/3187540461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9601008f0 0x7fd960100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- 172.21.15.31:0/3187540461 >> 172.21.15.31:0/3187540461 conn(0x7fd9600777a0 msgr2=0x7fd9600feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- 172.21.15.31:0/3187540461 shutdown_connections 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- 172.21.15.31:0/3187540461 wait complete. 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 Processor -- start 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- start start 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9601008f0 0x7fd9600fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd9600fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9601068a0 0x7fd9600fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd960110250 con 0x7fd9601012b0 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601100d0 con 0x7fd9601008f0 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.815+0000 7fd96769c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601103d0 con 0x7fd9601068a0 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd9600fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd9600fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32772/0 (socket says 172.21.15.31:32772) 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 -- 172.21.15.31:0/3180311291 learned_addr learned my addr 172.21.15.31:0/3180311291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd965438700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9601008f0 0x7fd9600fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd965c39700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9601068a0 0x7fd9600fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 -- 172.21.15.31:0/3180311291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9601068a0 msgr2=0x7fd9600fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9601068a0 0x7fd9600fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 -- 172.21.15.31:0/3180311291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9601008f0 msgr2=0x7fd9600fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9601008f0 0x7fd9600fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 -- 172.21.15.31:0/3180311291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9601a6dd0 con 0x7fd9601012b0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd964c37700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd9600fd150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd95000b530 tx=0x7fd95000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950015020 con 0x7fd9601012b0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd950011ce0 con 0x7fd9601012b0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9601a70c0 con 0x7fd9601012b0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950010e40 con 0x7fd9601012b0 2024-04-25T00:28:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.816+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9601a7650 con 0x7fd9601012b0 2024-04-25T00:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.817+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd960101fe0 con 0x7fd9601012b0 2024-04-25T00:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.819+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd950018020 con 0x7fd9601012b0 2024-04-25T00:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.820+0000 7fd9567fc700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd94c0a2d10 0x7fd94c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.820+0000 7fd965438700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd94c0a2d10 0x7fd94c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.820+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd9500ceda0 con 0x7fd9601012b0 2024-04-25T00:28:42.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.820+0000 7fd965438700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd94c0a2d10 0x7fd94c0a51d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00ac50 tx=0x7fd95c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:42.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:42.823+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd95001f020 con 0x7fd9601012b0 2024-04-25T00:28:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.033+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9600fe140 con 0x7fd9601012b0 2024-04-25T00:28:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.034+0000 7fd9567fc700 1 -- 172.21.15.31:0/3180311291 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22209 (secure 0 0 0) 0x7fd950010380 con 0x7fd9601012b0 2024-04-25T00:28:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:28:43.036 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:28:25.157023+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:28:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.036+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd94c0a2d10 msgr2=0x7fd94c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.036+0000 7fd96769c700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd94c0a2d10 0x7fd94c0a51d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00ac50 tx=0x7fd95c00a380 comp rx=0 tx=0).stop 2024-04-25T00:28:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.036+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 msgr2=0x7fd9600fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.036+0000 7fd96769c700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd9600fd150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd95000b530 tx=0x7fd95000b840 comp rx=0 tx=0).stop 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 shutdown_connections 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9601068a0 0x7fd9600fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9601008f0 0x7fd9600fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd94c0a2d10 0x7fd94c0a51d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 --2- 172.21.15.31:0/3180311291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9601012b0 0x7fd9600fd150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 >> 172.21.15.31:0/3180311291 conn(0x7fd9600777a0 msgr2=0x7fd960103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 shutdown_connections 2024-04-25T00:28:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:43.037+0000 7fd96769c700 1 -- 172.21.15.31:0/3180311291 wait complete. 2024-04-25T00:28:43.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:43 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3180311291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:43 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3180311291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:43.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:43 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3180311291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:28:43.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T00:28:43.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T00:28:43.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T00:28:43.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T00:28:43.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T00:28:43.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T00:28:43.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T00:28:43.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T00:28:43.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T00:28:43.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T00:28:43.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T00:28:43.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T00:28:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.602 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.602 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.609 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:28:44.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:44 smithi169 ceph-mon[33112]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 KiB/s wr, 24 op/s 2024-04-25T00:28:44.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:44 smithi040 ceph-mon[32945]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 KiB/s wr, 24 op/s 2024-04-25T00:28:44.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:44 smithi031 ceph-mon[29167]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 KiB/s wr, 24 op/s 2024-04-25T00:28:46.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:46 smithi031 ceph-mon[29167]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-25T00:28:46.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:46 smithi169 ceph-mon[33112]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-25T00:28:46.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:46 smithi040 ceph-mon[32945]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-25T00:28:48.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:48 smithi031 ceph-mon[29167]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-25T00:28:48.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:48 smithi169 ceph-mon[33112]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-25T00:28:48.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:48 smithi040 ceph-mon[32945]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-25T00:28:50.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:50 smithi031 ceph-mon[29167]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T00:28:50.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:50 smithi169 ceph-mon[33112]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T00:28:50.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:50 smithi040 ceph-mon[32945]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T00:28:52.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:52 smithi040 ceph-mon[32945]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T00:28:52.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:52 smithi031 ceph-mon[29167]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T00:28:52.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:52 smithi169 ceph-mon[33112]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T00:28:54.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:54 smithi169 ceph-mon[33112]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T00:28:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:54 smithi031 ceph-mon[29167]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T00:28:54.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:54 smithi040 ceph-mon[32945]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T00:28:56.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:56 smithi040 ceph-mon[32945]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:28:56.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:56 smithi031 ceph-mon[29167]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:28:56.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:56 smithi169 ceph-mon[33112]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:28:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/590189868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 msgr2=0x7f8c08074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/590189868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c08074840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009990 tx=0x7f8bfc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/590189868 shutdown_connections 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/590189868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c08074840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/590189868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c08113400 0x7f8c081157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/590189868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c08074d80 0x7f8c08112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/590189868 >> 172.21.15.31:0/590189868 conn(0x7f8c08079390 msgr2=0x7f8c080797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/590189868 shutdown_connections 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/590189868 wait complete. 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 Processor -- start 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.893+0000 7f8c0ef3d700 1 -- start start 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c0819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c08074d80 0x7f8c0819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c08113400 0x7f8c081a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08118a10 con 0x7f8c08074460 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08118890 con 0x7f8c08074d80 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08118b90 con 0x7f8c08113400 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c0819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c0819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51282/0 (socket says 172.21.15.31:51282) 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 -- 172.21.15.31:0/1740172389 learned_addr learned my addr 172.21.15.31:0/1740172389 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0d4da700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c08113400 0x7f8c081a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c07fff700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c08074d80 0x7f8c0819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 -- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c08113400 msgr2=0x7f8c081a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c08113400 0x7f8c081a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 -- 172.21.15.31:0/1740172389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c08074d80 msgr2=0x7f8c0819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c08074d80 0x7f8c0819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 -- 172.21.15.31:0/1740172389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c081a3f50 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ccd9700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c0819ef40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc027070 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c081a41e0 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bfc022470 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.894+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc017650 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.895+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c081abac0 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.895+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c08112d00 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.897+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8bfc02a070 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.897+0000 7f8c05ffb700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bf80a2cb0 0x7f8bf80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.897+0000 7f8c07fff700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bf80a2cb0 0x7f8bf80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.897+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8bfc0d0610 con 0x7f8c08074460 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.897+0000 7f8c05ffb700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] conn(0x7f8bf80ac490 0x7f8bf80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.898+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 --> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bfc0095d0 con 0x7f8bf80ac490 2024-04-25T00:28:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.898+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc0d0af0 con 0x7f8c08074460 2024-04-25T00:28:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.898+0000 7f8c07fff700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bf80a2cb0 0x7f8bf80a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c0810fe00 tx=0x7f8bf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.898+0000 7f8c0d4da700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] conn(0x7f8bf80ac490 0x7f8bf80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.898+0000 7f8c0d4da700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] conn(0x7f8bf80ac490 0x7f8bf80ae8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.899+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== osd.5 v2:172.21.15.169:6808/4050548275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bfc0095d0 con 0x7f8bf80ac490 2024-04-25T00:28:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.916+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 --> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c08075b50 con 0x7f8bf80ac490 2024-04-25T00:28:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.916+0000 7f8c05ffb700 1 -- 172.21.15.31:0/1740172389 <== osd.5 v2:172.21.15.169:6808/4050548275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c08075b50 con 0x7f8bf80ac490 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] conn(0x7f8bf80ac490 msgr2=0x7f8bf80ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] conn(0x7f8bf80ac490 0x7f8bf80ae8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bf80a2cb0 msgr2=0x7f8bf80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bf80a2cb0 0x7f8bf80a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c0810fe00 tx=0x7f8bf4009260 comp rx=0 tx=0).stop 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 msgr2=0x7f8c0819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 --2- 172.21.15.31:0/1740172389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c08074460 0x7f8c0819ef40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc0047b0 comp rx=0 tx=0).stop 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0d4da700 1 -- 172.21.15.31:0/1740172389 reap_dead start 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 shutdown_connections 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 >> 172.21.15.31:0/1740172389 conn(0x7f8c08079390 msgr2=0x7f8c081122a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 shutdown_connections 2024-04-25T00:28:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.917+0000 7f8c0ef3d700 1 -- 172.21.15.31:0/1740172389 wait complete. 2024-04-25T00:28:58.618 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:28:58 smithi031 ceph-mon[29167]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:28:58.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:28:58 smithi169 ceph-mon[33112]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:28:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:28:58 smithi040 ceph-mon[32945]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:28:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 -- 172.21.15.31:0/2927439003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 msgr2=0x7f9034106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 --2- 172.21.15.31:0/2927439003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f9034106e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9024009990 tx=0x7f9024009ca0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 -- 172.21.15.31:0/2927439003 shutdown_connections 2024-04-25T00:28:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 --2- 172.21.15.31:0/2927439003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9034107570 0x7f903410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 --2- 172.21.15.31:0/2927439003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90341024e0 0x7f90341028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 --2- 172.21.15.31:0/2927439003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f9034106e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 -- 172.21.15.31:0/2927439003 >> 172.21.15.31:0/2927439003 conn(0x7f9034079390 msgr2=0x7f90340797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 -- 172.21.15.31:0/2927439003 shutdown_connections 2024-04-25T00:28:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.355+0000 7f90390b3700 1 -- 172.21.15.31:0/2927439003 wait complete. 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 Processor -- start 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 -- start start 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90341024e0 0x7f90341a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f90341a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9034107570 0x7f903419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9034112440 con 0x7f9034102e90 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90341122c0 con 0x7f9034107570 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.356+0000 7f90390b3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90341125c0 con 0x7f90341024e0 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f90341a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f90341a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51310/0 (socket says 172.21.15.31:51310) 2024-04-25T00:28:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 -- 172.21.15.31:0/1720791278 learned_addr learned my addr 172.21.15.31:0/1720791278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f9032d9d700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90341024e0 0x7f90341a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903359e700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9034107570 0x7f903419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 -- 172.21.15.31:0/1720791278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90341024e0 msgr2=0x7f90341a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90341024e0 0x7f90341a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 -- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9034107570 msgr2=0x7f903419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9034107570 0x7f903419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 -- 172.21.15.31:0/1720791278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903419b120 con 0x7f9034102e90 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f9032d9d700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90341024e0 0x7f90341a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903359e700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9034107570 0x7f903419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f903259c700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f90341a0d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9024000c00 tx=0x7f9024003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024027070 con 0x7f9034102e90 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f902400bdc0 con 0x7f9034102e90 2024-04-25T00:28:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903419b3b0 con 0x7f9034102e90 2024-04-25T00:28:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902400f460 con 0x7f9034102e90 2024-04-25T00:28:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.357+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90341a76d0 con 0x7f9034102e90 2024-04-25T00:28:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.358+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90340757c0 con 0x7f9034102e90 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f902402a070 con 0x7f9034102e90 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f901bfff700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f901c0a2d30 0x7f901c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f9032d9d700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f901c0a2d30 0x7f901c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f90240d0370 con 0x7f9034102e90 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f901bfff700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] conn(0x7f901c0ac510 0x7f901c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 --> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90240043d0 con 0x7f901c0ac510 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.360+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f90240d0850 con 0x7f9034102e90 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.361+0000 7f9032d9d700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f901c0a2d30 0x7f901c0a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9028005950 tx=0x7f9028005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.361+0000 7f903359e700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] conn(0x7f901c0ac510 0x7f901c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.361+0000 7f903359e700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] conn(0x7f901c0ac510 0x7f901c0ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.362+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== osd.7 v2:172.21.15.40:6816/4053415057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90240043d0 con 0x7f901c0ac510 2024-04-25T00:28:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.378+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 --> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9034103970 con 0x7f901c0ac510 2024-04-25T00:28:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f901bfff700 1 -- 172.21.15.31:0/1720791278 <== osd.7 v2:172.21.15.40:6816/4053415057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9034103970 con 0x7f901c0ac510 2024-04-25T00:28:59.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] conn(0x7f901c0ac510 msgr2=0x7f901c0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] conn(0x7f901c0ac510 0x7f901c0ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f901c0a2d30 msgr2=0x7f901c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f901c0a2d30 0x7f901c0a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9028005950 tx=0x7f9028005f50 comp rx=0 tx=0).stop 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 msgr2=0x7f90341a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 --2- 172.21.15.31:0/1720791278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9034102e90 0x7f90341a0d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9024000c00 tx=0x7f9024003b30 comp rx=0 tx=0).stop 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f903359e700 1 -- 172.21.15.31:0/1720791278 reap_dead start 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 shutdown_connections 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.379+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 >> 172.21.15.31:0/1720791278 conn(0x7f9034079390 msgr2=0x7f9034100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.380+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 shutdown_connections 2024-04-25T00:28:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.380+0000 7f90390b3700 1 -- 172.21.15.31:0/1720791278 wait complete. 2024-04-25T00:28:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 -- 172.21.15.31:0/626075664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41068a0 msgr2=0x7f2db4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 --2- 172.21.15.31:0/626075664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41068a0 0x7f2db4106c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009990 tx=0x7f2da4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 -- 172.21.15.31:0/626075664 shutdown_connections 2024-04-25T00:28:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 --2- 172.21.15.31:0/626075664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41068a0 0x7f2db4106c80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 --2- 172.21.15.31:0/626075664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db41012b0 0x7f2db410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 --2- 172.21.15.31:0/626075664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db41008f0 0x7f2db4100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 -- 172.21.15.31:0/626075664 >> 172.21.15.31:0/626075664 conn(0x7f2db40777a0 msgr2=0x7f2db40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 -- 172.21.15.31:0/626075664 shutdown_connections 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.565+0000 7f2db99a9700 1 -- 172.21.15.31:0/626075664 wait complete. 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 Processor -- start 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 -- start start 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db41008f0 0x7f2db419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41012b0 0x7f2db419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db41068a0 0x7f2db41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4110aa0 con 0x7f2db41012b0 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4110920 con 0x7f2db41068a0 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db99a9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4110c20 con 0x7f2db41008f0 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41012b0 0x7f2db419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41012b0 0x7f2db419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51322/0 (socket says 172.21.15.31:51322) 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 -- 172.21.15.31:0/2275967336 learned_addr learned my addr 172.21.15.31:0/2275967336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:28:59.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db2ffd700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db41008f0 0x7f2db419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db37fe700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db41068a0 0x7f2db41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 -- 172.21.15.31:0/2275967336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db41008f0 msgr2=0x7f2db419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2db41008f0 0x7f2db419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 -- 172.21.15.31:0/2275967336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db41068a0 msgr2=0x7f2db41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db41068a0 0x7f2db41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.566+0000 7f2db27fc700 1 -- 172.21.15.31:0/2275967336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db4198fc0 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.567+0000 7f2db27fc700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41012b0 0x7f2db419eb50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2da800c920 tx=0x7f2da800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.567+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8007b00 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.567+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db41992b0 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.567+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2da8004ce0 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.567+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8005560 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.567+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db41a5590 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.568+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2db40ff770 con 0x7f2db41012b0 2024-04-25T00:28:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.569+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2da8004750 con 0x7f2db41012b0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db89a7700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2d9c0a2cb0 0x7f2d9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db2ffd700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2d9c0a2cb0 0x7f2d9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f2da80cf160 con 0x7f2db41012b0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db89a7700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] conn(0x7f2d9c0ac490 0x7f2d9c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 --> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d9c0aef60 con 0x7f2d9c0ac490 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f2da80cf510 con 0x7f2db41012b0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db37fe700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] conn(0x7f2d9c0ac490 0x7f2d9c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db2ffd700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2d9c0a2cb0 0x7f2d9c0a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2da4006010 tx=0x7f2da401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.570+0000 7f2db37fe700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] conn(0x7f2d9c0ac490 0x7f2d9c0ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.571+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== osd.9 v2:172.21.15.31:6826/1961015503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d9c0aef60 con 0x7f2d9c0ac490 2024-04-25T00:28:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.588+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 --> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2db4101fe0 con 0x7f2d9c0ac490 2024-04-25T00:28:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.588+0000 7f2db89a7700 1 -- 172.21.15.31:0/2275967336 <== osd.9 v2:172.21.15.31:6826/1961015503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2db4101fe0 con 0x7f2d9c0ac490 2024-04-25T00:28:59.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] conn(0x7f2d9c0ac490 msgr2=0x7f2d9c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] conn(0x7f2d9c0ac490 0x7f2d9c0ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2d9c0a2cb0 msgr2=0x7f2d9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2d9c0a2cb0 0x7f2d9c0a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2da4006010 tx=0x7f2da401a040 comp rx=0 tx=0).stop 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41012b0 msgr2=0x7f2db419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 --2- 172.21.15.31:0/2275967336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db41012b0 0x7f2db419eb50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2da800c920 tx=0x7f2da800cce0 comp rx=0 tx=0).stop 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db37fe700 1 -- 172.21.15.31:0/2275967336 reap_dead start 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 shutdown_connections 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 >> 172.21.15.31:0/2275967336 conn(0x7f2db40777a0 msgr2=0x7f2db40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 shutdown_connections 2024-04-25T00:28:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:59.589+0000 7f2db99a9700 1 -- 172.21.15.31:0/2275967336 wait complete. 2024-04-25T00:29:00.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:00 smithi031 ceph-mon[29167]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:29:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/223399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca41064c0 msgr2=0x7fdca410e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/223399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca41064c0 0x7fdca410e060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0099c0 tx=0x7fdc8c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/223399498 shutdown_connections 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/223399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca41064c0 0x7fdca410e060 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/223399498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca4101770 0x7fdca4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/223399498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca4102120 0x7fdca4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/223399498 >> 172.21.15.31:0/223399498 conn(0x7fdca40780e0 msgr2=0x7fdca40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/223399498 shutdown_connections 2024-04-25T00:29:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.743+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/223399498 wait complete. 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 Processor -- start 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 -- start start 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca4101770 0x7fdca4102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca4102120 0x7fdca4103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca41064c0 0x7fdca41038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4111350 con 0x7fdca4102120 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41111d0 con 0x7fdca41064c0 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdcaaf4c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41114d0 con 0x7fdca4101770 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca4102120 0x7fdca4103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca4102120 0x7fdca4103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51344/0 (socket says 172.21.15.31:51344) 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdca3fff700 1 -- 172.21.15.31:0/2476667202 learned_addr learned my addr 172.21.15.31:0/2476667202 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdca8ce8700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca4101770 0x7fdca4102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.744+0000 7fdca94e9700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca41064c0 0x7fdca41038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca3fff700 1 -- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca4101770 msgr2=0x7fdca4102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca3fff700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdca4101770 0x7fdca4102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca3fff700 1 -- 172.21.15.31:0/2476667202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca41064c0 msgr2=0x7fdca41038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca3fff700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdca41064c0 0x7fdca41038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca3fff700 1 -- 172.21.15.31:0/2476667202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41a6ea0 con 0x7fdca4102120 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca3fff700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca4102120 0x7fdca4103370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400ada0 tx=0x7fdc9400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94017020 con 0x7fdca4102120 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc9400ce30 con 0x7fdca4102120 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41a7190 con 0x7fdca4102120 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94005480 con 0x7fdca4102120 2024-04-25T00:29:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.745+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca41a76d0 con 0x7fdca4102120 2024-04-25T00:29:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.746+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdca4062940 con 0x7fdca4102120 2024-04-25T00:29:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.747+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdc94004ad0 con 0x7fdca4102120 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.748+0000 7fdca1ffb700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc900a2d30 0x7fdc900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.748+0000 7fdca8ce8700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc900a2d30 0x7fdc900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.748+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fdc940cf200 con 0x7fdca4102120 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.748+0000 7fdca1ffb700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] conn(0x7fdc900ac570 0x7fdc900ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.748+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 --> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc94012e80 con 0x7fdc900ac570 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.748+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fdc940cf5b0 con 0x7fdca4102120 2024-04-25T00:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.749+0000 7fdca8ce8700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc900a2d30 0x7fdc900a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c005950 tx=0x7fdc9c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.749+0000 7fdca94e9700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] conn(0x7fdc900ac570 0x7fdc900ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.749+0000 7fdca94e9700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] conn(0x7fdc900ac570 0x7fdc900ae990 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.750+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== osd.11 v2:172.21.15.169:6824/3987371467 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc94012e80 con 0x7fdc900ac570 2024-04-25T00:29:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.767+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 --> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdca4062dc0 con 0x7fdc900ac570 2024-04-25T00:29:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.768+0000 7fdca1ffb700 1 -- 172.21.15.31:0/2476667202 <== osd.11 v2:172.21.15.169:6824/3987371467 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdca4062dc0 con 0x7fdc900ac570 2024-04-25T00:29:00.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.768+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] conn(0x7fdc900ac570 msgr2=0x7fdc900ae990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:00.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.768+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] conn(0x7fdc900ac570 0x7fdc900ae990 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.768+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc900a2d30 msgr2=0x7fdc900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:00.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdc900a2d30 0x7fdc900a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c005950 tx=0x7fdc9c0058e0 comp rx=0 tx=0).stop 2024-04-25T00:29:00.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca4102120 msgr2=0x7fdca4103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:00.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 --2- 172.21.15.31:0/2476667202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca4102120 0x7fdca4103370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400ada0 tx=0x7fdc9400c330 comp rx=0 tx=0).stop 2024-04-25T00:29:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdca94e9700 1 -- 172.21.15.31:0/2476667202 reap_dead start 2024-04-25T00:29:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 shutdown_connections 2024-04-25T00:29:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 >> 172.21.15.31:0/2476667202 conn(0x7fdca40780e0 msgr2=0x7fdca406ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 shutdown_connections 2024-04-25T00:29:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:00.769+0000 7fdcaaf4c700 1 -- 172.21.15.31:0/2476667202 wait complete. 2024-04-25T00:29:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:00 smithi169 ceph-mon[33112]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:29:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:00 smithi040 ceph-mon[32945]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:29:01.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 -- 172.21.15.31:0/2989445788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 msgr2=0x7f4b8c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 --2- 172.21.15.31:0/2989445788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c1018b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4b800099c0 tx=0x7f4b80009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 -- 172.21.15.31:0/2989445788 shutdown_connections 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 --2- 172.21.15.31:0/2989445788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b8c101df0 0x7f4b8c10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 --2- 172.21.15.31:0/2989445788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c107260 0x7f4b8c107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 --2- 172.21.15.31:0/2989445788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c1018b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 -- 172.21.15.31:0/2989445788 >> 172.21.15.31:0/2989445788 conn(0x7f4b8c077f00 msgr2=0x7f4b8c078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 -- 172.21.15.31:0/2989445788 shutdown_connections 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.781+0000 7f4b92f86700 1 -- 172.21.15.31:0/2989445788 wait complete. 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 Processor -- start 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 -- start start 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b8c101df0 0x7f4b8c103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c107260 0x7f4b8c1038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c111300 con 0x7f4b8c101450 2024-04-25T00:29:01.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c111180 con 0x7f4b8c101df0 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b92f86700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c111480 con 0x7f4b8c107260 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.782+0000 7f4b90d22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51270/0 (socket says 172.21.15.31:51270) 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 -- 172.21.15.31:0/628553578 learned_addr learned my addr 172.21.15.31:0/628553578 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b8bfff700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b8c101df0 0x7f4b8c103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b91523700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c107260 0x7f4b8c1038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 -- 172.21.15.31:0/628553578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c107260 msgr2=0x7f4b8c1038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c107260 0x7f4b8c1038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 -- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b8c101df0 msgr2=0x7f4b8c103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b8c101df0 0x7f4b8c103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 -- 172.21.15.31:0/628553578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c1a6e50 con 0x7f4b8c101450 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b8bfff700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b8c101df0 0x7f4b8c103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b91523700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c107260 0x7f4b8c1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:01.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b90d22700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c102e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c007f10 tx=0x7f4b7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00dc90 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b7c00de30 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c1a7140 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0133f0 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.783+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1a7650 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.784+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b8c106200 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.786+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4b7c01a020 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.786+0000 7f4b89ffb700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b740a2d30 0x7f4b740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b8bfff700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b740a2d30 0x7f4b740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f4b7c0ceef0 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b89ffb700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] conn(0x7f4b740ac510 0x7f4b740ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 --> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b7c005390 con 0x7f4b740ac510 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f4b7c0cf360 con 0x7f4b8c101450 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b8bfff700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b740a2d30 0x7f4b740a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c1048b0 tx=0x7f4b80005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.787+0000 7f4b91523700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] conn(0x7f4b740ac510 0x7f4b740ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.788+0000 7f4b91523700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] conn(0x7f4b740ac510 0x7f4b740ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.789+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== osd.10 v2:172.21.15.40:6824/3974437906 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b7c005390 con 0x7f4b740ac510 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.805+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 --> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4b8c1028c0 con 0x7f4b740ac510 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b89ffb700 1 -- 172.21.15.31:0/628553578 <== osd.10 v2:172.21.15.40:6824/3974437906 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4b8c1028c0 con 0x7f4b740ac510 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] conn(0x7f4b740ac510 msgr2=0x7f4b740ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b92f86700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] conn(0x7f4b740ac510 0x7f4b740ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b740a2d30 msgr2=0x7f4b740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b92f86700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4b740a2d30 0x7f4b740a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c1048b0 tx=0x7f4b80005fd0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 msgr2=0x7f4b8c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.806+0000 7f4b92f86700 1 --2- 172.21.15.31:0/628553578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b8c101450 0x7f4b8c102e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c007f10 tx=0x7f4b7c00d3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.807+0000 7f4b91523700 1 -- 172.21.15.31:0/628553578 reap_dead start 2024-04-25T00:29:01.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.807+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 shutdown_connections 2024-04-25T00:29:01.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.807+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 >> 172.21.15.31:0/628553578 conn(0x7f4b8c077f00 msgr2=0x7f4b8c06ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:01.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.807+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 shutdown_connections 2024-04-25T00:29:01.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:28:57.807+0000 7f4b92f86700 1 -- 172.21.15.31:0/628553578 wait complete. 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 -- 172.21.15.31:0/3415598112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102ec0 msgr2=0x7f88dc106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/3415598112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102ec0 0x7f88dc106eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009990 tx=0x7f88cc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 -- 172.21.15.31:0/3415598112 shutdown_connections 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/3415598112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88dc107590 0x7f88dc10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/3415598112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc102510 0x7f88dc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/3415598112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102ec0 0x7f88dc106eb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 -- 172.21.15.31:0/3415598112 >> 172.21.15.31:0/3415598112 conn(0x7f88dc0793a0 msgr2=0x7f88dc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 -- 172.21.15.31:0/3415598112 shutdown_connections 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.669+0000 7f88e3c8b700 1 -- 172.21.15.31:0/3415598112 wait complete. 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 Processor -- start 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 -- start start 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102510 0x7f88dc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88dc102ec0 0x7f88dc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc107590 0x7f88dc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1123e0 con 0x7f88dc102510 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc112260 con 0x7f88dc102ec0 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e3c8b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc112560 con 0x7f88dc107590 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e1a27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102510 0x7f88dc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e1a27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102510 0x7f88dc19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51368/0 (socket says 172.21.15.31:51368) 2024-04-25T00:29:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e1a27700 1 -- 172.21.15.31:0/234917546 learned_addr learned my addr 172.21.15.31:0/234917546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e1226700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88dc102ec0 0x7f88dc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.670+0000 7f88e2228700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc107590 0x7f88dc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e1a27700 1 -- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc107590 msgr2=0x7f88dc198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e1a27700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc107590 0x7f88dc198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e1a27700 1 -- 172.21.15.31:0/234917546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88dc102ec0 msgr2=0x7f88dc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e1a27700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88dc102ec0 0x7f88dc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e1a27700 1 -- 172.21.15.31:0/234917546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc198f60 con 0x7f88dc102510 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e1a27700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102510 0x7f88dc19e5b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c9b0 tx=0x7f88d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8007b00 con 0x7f88dc102510 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88d800ce50 con 0x7f88dc102510 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc199250 con 0x7f88dc102510 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8018660 con 0x7f88dc102510 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.671+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc1a5530 con 0x7f88dc102510 2024-04-25T00:29:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.672+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88dc0629c0 con 0x7f88dc102510 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.673+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f88d800f450 con 0x7f88dc102510 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88d2ffd700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f88c80a2d30 0x7f88c80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88e1226700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f88c80a2d30 0x7f88c80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f88d80cf310 con 0x7f88dc102510 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88d2ffd700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] conn(0x7f88c80ac510 0x7f88c80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 --> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88d8019ec0 con 0x7f88c80ac510 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f88d80cf7f0 con 0x7f88dc102510 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.674+0000 7f88e1226700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f88c80a2d30 0x7f88c80a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b5c0 tx=0x7f88cc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.675+0000 7f88e2228700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] conn(0x7f88c80ac510 0x7f88c80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.675+0000 7f88e2228700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] conn(0x7f88c80ac510 0x7f88c80ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.676+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== osd.8 v2:172.21.15.169:6816/3220692863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88d8019ec0 con 0x7f88c80ac510 2024-04-25T00:29:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.692+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 --> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88dc062e40 con 0x7f88c80ac510 2024-04-25T00:29:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88d2ffd700 1 -- 172.21.15.31:0/234917546 <== osd.8 v2:172.21.15.169:6816/3220692863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f88dc062e40 con 0x7f88c80ac510 2024-04-25T00:29:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] conn(0x7f88c80ac510 msgr2=0x7f88c80ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] conn(0x7f88c80ac510 0x7f88c80ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f88c80a2d30 msgr2=0x7f88c80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f88c80a2d30 0x7f88c80a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b5c0 tx=0x7f88cc005960 comp rx=0 tx=0).stop 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102510 msgr2=0x7f88dc19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 --2- 172.21.15.31:0/234917546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88dc102510 0x7f88dc19e5b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c9b0 tx=0x7f88d800ccc0 comp rx=0 tx=0).stop 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e2228700 1 -- 172.21.15.31:0/234917546 reap_dead start 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 shutdown_connections 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.693+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 >> 172.21.15.31:0/234917546 conn(0x7f88dc0793a0 msgr2=0x7f88dc100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.694+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 shutdown_connections 2024-04-25T00:29:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:01.694+0000 7f88e3c8b700 1 -- 172.21.15.31:0/234917546 wait complete. 2024-04-25T00:29:02.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:02 smithi031 ceph-mon[29167]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:02.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:02 smithi169 ceph-mon[33112]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:02 smithi040 ceph-mon[32945]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.288+0000 7fdd19f21700 1 -- 172.21.15.31:0/369362601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd141017e0 msgr2=0x7fdd14106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.288+0000 7fdd19f21700 1 --2- 172.21.15.31:0/369362601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd141017e0 0x7fdd14106f20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd080099c0 tx=0x7fdd08009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.288+0000 7fdd19f21700 1 -- 172.21.15.31:0/369362601 shutdown_connections 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.288+0000 7fdd19f21700 1 --2- 172.21.15.31:0/369362601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd14107650 0x7fdd1410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.288+0000 7fdd19f21700 1 --2- 172.21.15.31:0/369362601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd14100e30 0x7fdd14101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.288+0000 7fdd19f21700 1 --2- 172.21.15.31:0/369362601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd141017e0 0x7fdd14106f20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.289+0000 7fdd19f21700 1 -- 172.21.15.31:0/369362601 >> 172.21.15.31:0/369362601 conn(0x7fdd14076450 msgr2=0x7fdd14076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.289+0000 7fdd19f21700 1 -- 172.21.15.31:0/369362601 shutdown_connections 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.289+0000 7fdd19f21700 1 -- 172.21.15.31:0/369362601 wait complete. 2024-04-25T00:29:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.289+0000 7fdd19f21700 1 Processor -- start 2024-04-25T00:29:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.289+0000 7fdd19f21700 1 -- start start 2024-04-25T00:29:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd19f21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd14100e30 0x7fdd14101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd19f21700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd141017e0 0x7fdd141022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd19f21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14107650 0x7fdd14102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd19f21700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14112410 con 0x7fdd14107650 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd19f21700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14112290 con 0x7fdd14100e30 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd19f21700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14112590 con 0x7fdd141017e0 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14107650 0x7fdd14102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14107650 0x7fdd14102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36630/0 (socket says 172.21.15.31:36630) 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 -- 172.21.15.31:0/815603948 learned_addr learned my addr 172.21.15.31:0/815603948 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd12ffd700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd141017e0 0x7fdd141022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd137fe700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd14100e30 0x7fdd14101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 -- 172.21.15.31:0/815603948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd141017e0 msgr2=0x7fdd141022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd141017e0 0x7fdd141022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 -- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd14100e30 msgr2=0x7fdd14101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd14100e30 0x7fdd14101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 -- 172.21.15.31:0/815603948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd141a6df0 con 0x7fdd14107650 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd137fe700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd14100e30 0x7fdd14101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd13fff700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14107650 0x7fdd14102830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007ee0 tx=0x7fdd0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c00dbd0 con 0x7fdd14107650 2024-04-25T00:29:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.290+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd0c00dd70 con 0x7fdd14107650 2024-04-25T00:29:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.291+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd141a70e0 con 0x7fdd14107650 2024-04-25T00:29:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.291+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a75a0 con 0x7fdd14107650 2024-04-25T00:29:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.291+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c005990 con 0x7fdd14107650 2024-04-25T00:29:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.291+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd00001030 con 0x7fdd14107650 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.293+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdd0c01a020 con 0x7fdd14107650 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.293+0000 7fdd10ff9700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdcfc0a2cb0 0x7fdcfc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd137fe700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdcfc0a2cb0 0x7fdcfc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fdd0c0cecb0 con 0x7fdd14107650 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd10ff9700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] conn(0x7fdcfc0ac490 0x7fdcfc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 --> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdcfc0aef60 con 0x7fdcfc0ac490 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fdd0c0cf190 con 0x7fdd14107650 2024-04-25T00:29:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd137fe700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdcfc0a2cb0 0x7fdcfc0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005fd0 tx=0x7fdd04005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.294+0000 7fdd12ffd700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] conn(0x7fdcfc0ac490 0x7fdcfc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.295+0000 7fdd12ffd700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] conn(0x7fdcfc0ac490 0x7fdcfc0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.295+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== osd.1 v2:172.21.15.40:6800/2453389189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdcfc0aef60 con 0x7fdcfc0ac490 2024-04-25T00:29:04.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.314+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 --> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd00002e60 con 0x7fdcfc0ac490 2024-04-25T00:29:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.314+0000 7fdd10ff9700 1 -- 172.21.15.31:0/815603948 <== osd.1 v2:172.21.15.40:6800/2453389189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdd00002e60 con 0x7fdcfc0ac490 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] conn(0x7fdcfc0ac490 msgr2=0x7fdcfc0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] conn(0x7fdcfc0ac490 0x7fdcfc0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdcfc0a2cb0 msgr2=0x7fdcfc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdcfc0a2cb0 0x7fdcfc0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005fd0 tx=0x7fdd04005ee0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14107650 msgr2=0x7fdd14102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 --2- 172.21.15.31:0/815603948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14107650 0x7fdd14102830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007ee0 tx=0x7fdd0c00d3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd13fff700 1 -- 172.21.15.31:0/815603948 reap_dead start 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 shutdown_connections 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 >> 172.21.15.31:0/815603948 conn(0x7fdd14076450 msgr2=0x7fdd140fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 shutdown_connections 2024-04-25T00:29:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:04.315+0000 7fdd19f21700 1 -- 172.21.15.31:0/815603948 wait complete. 2024-04-25T00:29:04.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:04 smithi031 ceph-mon[29167]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T00:29:04.776 INFO:teuthology.orchestra.run.smithi031.stdout:227633266729 2024-04-25T00:29:04.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T00:29:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:04 smithi169 ceph-mon[33112]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T00:29:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:04 smithi040 ceph-mon[32945]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T00:29:05.711 INFO:teuthology.orchestra.run.smithi031.stdout:296352743454 2024-04-25T00:29:05.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T00:29:06.456 INFO:teuthology.orchestra.run.smithi031.stdout:317827579930 2024-04-25T00:29:06.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.206+0000 7f277b620700 1 -- 172.21.15.31:0/1197636065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2774101e40 msgr2=0x7f277410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.206+0000 7f277b620700 1 --2- 172.21.15.31:0/1197636065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2774101e40 0x7f277410dfc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f276c00b0d0 tx=0x7f276c00b3e0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 -- 172.21.15.31:0/1197636065 shutdown_connections 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 --2- 172.21.15.31:0/1197636065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2774101e40 0x7f277410dfc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 --2- 172.21.15.31:0/1197636065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27741061b0 0x7f2774106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 --2- 172.21.15.31:0/1197636065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2774101480 0x7f2774101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 -- 172.21.15.31:0/1197636065 >> 172.21.15.31:0/1197636065 conn(0x7f2774076fa0 msgr2=0x7f27740773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 -- 172.21.15.31:0/1197636065 shutdown_connections 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 -- 172.21.15.31:0/1197636065 wait complete. 2024-04-25T00:29:06.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 Processor -- start 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 -- start start 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2774101480 0x7f2774199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2774101e40 0x7f2774199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.207+0000 7f277b620700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27741061b0 0x7f27741a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f277b620700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f27741113d0 con 0x7f27741061b0 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f277b620700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2774111250 con 0x7f2774101480 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f277b620700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2774111550 con 0x7f2774101e40 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27741061b0 0x7f27741a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27741061b0 0x7f27741a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36650/0 (socket says 172.21.15.31:36650) 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 -- 172.21.15.31:0/977230073 learned_addr learned my addr 172.21.15.31:0/977230073 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2778bbb700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2774101e40 0x7f2774199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f27793bc700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2774101480 0x7f2774199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 -- 172.21.15.31:0/977230073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2774101e40 msgr2=0x7f2774199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2774101e40 0x7f2774199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 -- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2774101480 msgr2=0x7f2774199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2774101480 0x7f2774199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 -- 172.21.15.31:0/977230073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27741a4d50 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f2779bbd700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27741061b0 0x7f27741a4810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f276c00d010 tx=0x7f276c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c01f070 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f276c003c00 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27741a4f20 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c01b6a0 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.208+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27741a5370 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.209+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27740628c0 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.210+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f276c022070 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f276a7fc700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27600a2cb0 0x7f27600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f27793bc700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27600a2cb0 0x7f27600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f276c00f020 con 0x7f27741061b0 2024-04-25T00:29:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f276a7fc700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] conn(0x7f27600ac490 0x7f27600ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 --> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f276c01ae50 con 0x7f27600ac490 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f276c0d0830 con 0x7f27741061b0 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f27793bc700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27600a2cb0 0x7f27600a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2770005950 tx=0x7f2770005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.211+0000 7f2778bbb700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] conn(0x7f27600ac490 0x7f27600ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.212+0000 7f2778bbb700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] conn(0x7f27600ac490 0x7f27600ae8b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.213+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== osd.4 v2:172.21.15.40:6808/3167270680 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f276c01ae50 con 0x7f27600ac490 2024-04-25T00:29:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.229+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 --> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2774062d40 con 0x7f27600ac490 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f276a7fc700 1 -- 172.21.15.31:0/977230073 <== osd.4 v2:172.21.15.40:6808/3167270680 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2774062d40 con 0x7f27600ac490 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] conn(0x7f27600ac490 msgr2=0x7f27600ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] conn(0x7f27600ac490 0x7f27600ae8b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27600a2cb0 msgr2=0x7f27600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27600a2cb0 0x7f27600a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2770005950 tx=0x7f2770005c70 comp rx=0 tx=0).stop 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27741061b0 msgr2=0x7f27741a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 --2- 172.21.15.31:0/977230073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27741061b0 0x7f27741a4810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f276c00d010 tx=0x7f276c0079a0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f2779bbd700 1 -- 172.21.15.31:0/977230073 reap_dead start 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 shutdown_connections 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.230+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 >> 172.21.15.31:0/977230073 conn(0x7f2774076fa0 msgr2=0x7f27740ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.231+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 shutdown_connections 2024-04-25T00:29:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:05.231+0000 7f277b620700 1 -- 172.21.15.31:0/977230073 wait complete. 2024-04-25T00:29:06.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:06 smithi031 ceph-mon[29167]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:06 smithi169 ceph-mon[33112]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:06 smithi040 ceph-mon[32945]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:07.008 INFO:teuthology.orchestra.run.smithi031.stdout:261993005092 2024-04-25T00:29:07.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T00:29:07.663 INFO:teuthology.orchestra.run.smithi031.stdout:146028888122 2024-04-25T00:29:07.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T00:29:08.097 INFO:teuthology.orchestra.run.smithi031.stdout:352187318291 2024-04-25T00:29:08.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 -- 172.21.15.31:0/161158959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90105dd0 msgr2=0x7fcf90106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 --2- 172.21.15.31:0/161158959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90105dd0 0x7fcf90106230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009990 tx=0x7fcf78009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 -- 172.21.15.31:0/161158959 shutdown_connections 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 --2- 172.21.15.31:0/161158959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf90106770 0x7fcf9010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 --2- 172.21.15.31:0/161158959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf901016d0 0x7fcf90101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 --2- 172.21.15.31:0/161158959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90105dd0 0x7fcf90106230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 -- 172.21.15.31:0/161158959 >> 172.21.15.31:0/161158959 conn(0x7fcf90077ff0 msgr2=0x7fcf90078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 -- 172.21.15.31:0/161158959 shutdown_connections 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.373+0000 7fcf95a50700 1 -- 172.21.15.31:0/161158959 wait complete. 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.374+0000 7fcf95a50700 1 Processor -- start 2024-04-25T00:29:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.374+0000 7fcf95a50700 1 -- start start 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.374+0000 7fcf95a50700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf901016d0 0x7fcf9019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.374+0000 7fcf95a50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf90105dd0 0x7fcf9019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.374+0000 7fcf95a50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90106770 0x7fcf90198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf95a50700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90111300 con 0x7fcf90106770 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf95a50700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90111180 con 0x7fcf90105dd0 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf95a50700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90111480 con 0x7fcf901016d0 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90106770 0x7fcf90198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90106770 0x7fcf90198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36670/0 (socket says 172.21.15.31:36670) 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 -- 172.21.15.31:0/1301759962 learned_addr learned my addr 172.21.15.31:0/1301759962 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8e7fc700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf90105dd0 0x7fcf9019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8effd700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf901016d0 0x7fcf9019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 -- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf901016d0 msgr2=0x7fcf9019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf901016d0 0x7fcf9019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 -- 172.21.15.31:0/1301759962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf90105dd0 msgr2=0x7fcf9019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf90105dd0 0x7fcf9019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 -- 172.21.15.31:0/1301759962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90198d60 con 0x7fcf90106770 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.375+0000 7fcf8f7fe700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90106770 0x7fcf90198600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800ab50 tx=0x7fcf8800ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.376+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88017020 con 0x7fcf90106770 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.376+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf90199050 con 0x7fcf90106770 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.376+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf88004bb0 con 0x7fcf90106770 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.376+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88005350 con 0x7fcf90106770 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.376+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf901a5310 con 0x7fcf90106770 2024-04-25T00:29:08.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.376+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf900629c0 con 0x7fcf90106770 2024-04-25T00:29:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.379+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcf880054f0 con 0x7fcf90106770 2024-04-25T00:29:08.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.379+0000 7fcf94a4e700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcf7c0a2cb0 0x7fcf7c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:08.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.379+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fcf880cf390 con 0x7fcf90106770 2024-04-25T00:29:08.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.379+0000 7fcf8effd700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcf7c0a2cb0 0x7fcf7c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:08.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.380+0000 7fcf94a4e700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] conn(0x7fcf7c0ac490 0x7fcf7c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:08.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.380+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 --> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf8800fe50 con 0x7fcf7c0ac490 2024-04-25T00:29:08.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.380+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fcf880cf870 con 0x7fcf90106770 2024-04-25T00:29:08.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.380+0000 7fcf8effd700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcf7c0a2cb0 0x7fcf7c0a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000ac50 tx=0x7fcf8000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:08.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.380+0000 7fcf8e7fc700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] conn(0x7fcf7c0ac490 0x7fcf7c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:08.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.380+0000 7fcf8e7fc700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] conn(0x7fcf7c0ac490 0x7fcf7c0ae8b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:08.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.381+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== osd.2 v2:172.21.15.169:6800/1729358073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf8800fe50 con 0x7fcf7c0ac490 2024-04-25T00:29:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.399+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 --> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf90062e40 con 0x7fcf7c0ac490 2024-04-25T00:29:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.399+0000 7fcf94a4e700 1 -- 172.21.15.31:0/1301759962 <== osd.2 v2:172.21.15.169:6800/1729358073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcf90062e40 con 0x7fcf7c0ac490 2024-04-25T00:29:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] conn(0x7fcf7c0ac490 msgr2=0x7fcf7c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] conn(0x7fcf7c0ac490 0x7fcf7c0ae8b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcf7c0a2cb0 msgr2=0x7fcf7c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcf7c0a2cb0 0x7fcf7c0a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000ac50 tx=0x7fcf8000a380 comp rx=0 tx=0).stop 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90106770 msgr2=0x7fcf90198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 --2- 172.21.15.31:0/1301759962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf90106770 0x7fcf90198600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800ab50 tx=0x7fcf8800ae60 comp rx=0 tx=0).stop 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf8f7fe700 1 -- 172.21.15.31:0/1301759962 reap_dead start 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 shutdown_connections 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 >> 172.21.15.31:0/1301759962 conn(0x7fcf90077ff0 msgr2=0x7fcf9006ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 shutdown_connections 2024-04-25T00:29:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:08.400+0000 7fcf95a50700 1 -- 172.21.15.31:0/1301759962 wait complete. 2024-04-25T00:29:08.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:08 smithi031 ceph-mon[29167]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:08.821 INFO:teuthology.orchestra.run.smithi031.stdout:55834574921 2024-04-25T00:29:08.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T00:29:08.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:08 smithi169 ceph-mon[33112]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:08 smithi040 ceph-mon[32945]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 -- 172.21.15.31:0/3443254126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac1075a0 msgr2=0x7f37ac10f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 --2- 172.21.15.31:0/3443254126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac1075a0 0x7f37ac10f170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b0a0 tx=0x7f379c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 -- 172.21.15.31:0/3443254126 shutdown_connections 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 --2- 172.21.15.31:0/3443254126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac1075a0 0x7f37ac10f170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 --2- 172.21.15.31:0/3443254126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37ac102540 0x7f37ac102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 --2- 172.21.15.31:0/3443254126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac102ef0 0x7f37ac106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 -- 172.21.15.31:0/3443254126 >> 172.21.15.31:0/3443254126 conn(0x7f37ac0792c0 msgr2=0x7f37ac0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 -- 172.21.15.31:0/3443254126 shutdown_connections 2024-04-25T00:29:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.376+0000 7f37b173b700 1 -- 172.21.15.31:0/3443254126 wait complete. 2024-04-25T00:29:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 Processor -- start 2024-04-25T00:29:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 -- start start 2024-04-25T00:29:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac102540 0x7f37ac1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37ac102ef0 0x7f37ac1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac1075a0 0x7f37ac19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac112430 con 0x7f37ac102540 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1122b0 con 0x7f37ac102ef0 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.377+0000 7f37b173b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1125b0 con 0x7f37ac1075a0 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac102540 0x7f37ac1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac102540 0x7f37ac1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36688/0 (socket says 172.21.15.31:36688) 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 -- 172.21.15.31:0/4265697707 learned_addr learned my addr 172.21.15.31:0/4265697707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aa7fc700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37ac102ef0 0x7f37ac1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 -- 172.21.15.31:0/4265697707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac1075a0 msgr2=0x7f37ac19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37ab7fe700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac1075a0 0x7f37ac19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac1075a0 0x7f37ac19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 -- 172.21.15.31:0/4265697707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37ac102ef0 msgr2=0x7f37ac1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37ac102ef0 0x7f37ac1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 -- 172.21.15.31:0/4265697707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac19afc0 con 0x7f37ac102540 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37aaffd700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac102540 0x7f37ac1a06a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c9b0 tx=0x7f37a400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4007b00 con 0x7f37ac102540 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37a4007ca0 con 0x7f37ac102540 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac19b2b0 con 0x7f37ac102540 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4018660 con 0x7f37ac102540 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.378+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac1a7570 con 0x7f37ac102540 2024-04-25T00:29:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.379+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37ac0757c0 con 0x7f37ac102540 2024-04-25T00:29:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f37a400f450 con 0x7f37ac102540 2024-04-25T00:29:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37a3fff700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f37980a2cb0 0x7f37980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37aa7fc700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f37980a2cb0 0x7f37980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f37a40cf420 con 0x7f37ac102540 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37a3fff700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] conn(0x7f37980ac490 0x7f37980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 --> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37980aef60 con 0x7f37980ac490 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.381+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f37a40cf900 con 0x7f37ac102540 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.382+0000 7f37ab7fe700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] conn(0x7f37980ac490 0x7f37980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.382+0000 7f37aa7fc700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f37980a2cb0 0x7f37980a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f37ac10f520 tx=0x7f379400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.382+0000 7f37ab7fe700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] conn(0x7f37980ac490 0x7f37980ae8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.383+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== osd.6 v2:172.21.15.31:6818/3697284546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37980aef60 con 0x7f37980ac490 2024-04-25T00:29:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.400+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 --> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37ac1039a0 con 0x7f37980ac490 2024-04-25T00:29:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.400+0000 7f37a3fff700 1 -- 172.21.15.31:0/4265697707 <== osd.6 v2:172.21.15.31:6818/3697284546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37ac1039a0 con 0x7f37980ac490 2024-04-25T00:29:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.400+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] conn(0x7f37980ac490 msgr2=0x7f37980ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.400+0000 7f37b173b700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] conn(0x7f37980ac490 0x7f37980ae8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.400+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f37980a2cb0 msgr2=0x7f37980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.400+0000 7f37b173b700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f37980a2cb0 0x7f37980a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f37ac10f520 tx=0x7f379400a300 comp rx=0 tx=0).stop 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac102540 msgr2=0x7f37ac1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37b173b700 1 --2- 172.21.15.31:0/4265697707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac102540 0x7f37ac1a06a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c9b0 tx=0x7f37a400ccc0 comp rx=0 tx=0).stop 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37ab7fe700 1 -- 172.21.15.31:0/4265697707 reap_dead start 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 shutdown_connections 2024-04-25T00:29:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 >> 172.21.15.31:0/4265697707 conn(0x7f37ac0792c0 msgr2=0x7f37ac100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 shutdown_connections 2024-04-25T00:29:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:09.401+0000 7f37b173b700 1 -- 172.21.15.31:0/4265697707 wait complete. 2024-04-25T00:29:10.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.165+0000 7fb99b5a8700 1 -- 172.21.15.31:0/4238790204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 msgr2=0x7fb994105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:10.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.165+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/4238790204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb994105d70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9840099c0 tx=0x7fb984009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 -- 172.21.15.31:0/4238790204 shutdown_connections 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/4238790204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb994106450 0x7fb99410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/4238790204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013c0 0x7fb9941017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/4238790204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb994105d70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 -- 172.21.15.31:0/4238790204 >> 172.21.15.31:0/4238790204 conn(0x7fb9940780e0 msgr2=0x7fb9940784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 -- 172.21.15.31:0/4238790204 shutdown_connections 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 -- 172.21.15.31:0/4238790204 wait complete. 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 Processor -- start 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.166+0000 7fb99b5a8700 1 -- start start 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb99b5a8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9941013c0 0x7fb9941a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb99b5a8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb9941a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb99b5a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb994106450 0x7fb99419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb99b5a8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb994111360 con 0x7fb994101d70 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb99b5a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9941111e0 con 0x7fb994106450 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb99b5a8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9941114e0 con 0x7fb9941013c0 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb9941a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb9941a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36710/0 (socket says 172.21.15.31:36710) 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 -- 172.21.15.31:0/3840021452 learned_addr learned my addr 172.21.15.31:0/3840021452 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb999344700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9941013c0 0x7fb9941a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb999b45700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb994106450 0x7fb99419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 -- 172.21.15.31:0/3840021452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9941013c0 msgr2=0x7fb9941a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9941013c0 0x7fb9941a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 -- 172.21.15.31:0/3840021452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb994106450 msgr2=0x7fb99419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb994106450 0x7fb99419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 -- 172.21.15.31:0/3840021452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99419b150 con 0x7fb994101d70 2024-04-25T00:29:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb999b45700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb994106450 0x7fb99419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb998b43700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb9941a0d70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb98400c010 tx=0x7fb9840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.167+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984027070 con 0x7fb994101d70 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.168+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99419b3e0 con 0x7fb994101d70 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.168+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb984003f50 con 0x7fb994101d70 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.168+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984022c70 con 0x7fb994101d70 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.168+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9941a7700 con 0x7fb994101d70 2024-04-25T00:29:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.168+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9940629c0 con 0x7fb994101d70 2024-04-25T00:29:10.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.170+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb98402a070 con 0x7fb994101d70 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.170+0000 7fb98a7fc700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb9800a2cb0 0x7fb9800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.170+0000 7fb999344700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb9800a2cb0 0x7fb9800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.170+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fb9840d0a10 con 0x7fb994101d70 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.171+0000 7fb98a7fc700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] conn(0x7fb9800ac490 0x7fb9800ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.171+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 --> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9840056d0 con 0x7fb9800ac490 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.171+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb9840d0ef0 con 0x7fb994101d70 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.171+0000 7fb999b45700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] conn(0x7fb9800ac490 0x7fb9800ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.171+0000 7fb999344700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb9800a2cb0 0x7fb9800a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb99419bb30 tx=0x7fb990009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.171+0000 7fb999b45700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] conn(0x7fb9800ac490 0x7fb9800ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:10.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.172+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== osd.3 v2:172.21.15.31:6810/2474264832 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9840056d0 con 0x7fb9800ac490 2024-04-25T00:29:10.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.189+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 --> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb994062dc0 con 0x7fb9800ac490 2024-04-25T00:29:10.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.189+0000 7fb98a7fc700 1 -- 172.21.15.31:0/3840021452 <== osd.3 v2:172.21.15.31:6810/2474264832 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb994062dc0 con 0x7fb9800ac490 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] conn(0x7fb9800ac490 msgr2=0x7fb9800ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] conn(0x7fb9800ac490 0x7fb9800ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb9800a2cb0 msgr2=0x7fb9800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fb9800a2cb0 0x7fb9800a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb99419bb30 tx=0x7fb990009190 comp rx=0 tx=0).stop 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 msgr2=0x7fb9941a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 --2- 172.21.15.31:0/3840021452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb994101d70 0x7fb9941a0d70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb98400c010 tx=0x7fb9840047b0 comp rx=0 tx=0).stop 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb999b45700 1 -- 172.21.15.31:0/3840021452 reap_dead start 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 shutdown_connections 2024-04-25T00:29:10.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 >> 172.21.15.31:0/3840021452 conn(0x7fb9940780e0 msgr2=0x7fb99410d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 shutdown_connections 2024-04-25T00:29:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:10.190+0000 7fb99b5a8700 1 -- 172.21.15.31:0/3840021452 wait complete. 2024-04-25T00:29:10.642 INFO:teuthology.orchestra.run.smithi031.stdout:201863462961 2024-04-25T00:29:10.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T00:29:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:10 smithi031 ceph-mon[29167]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T00:29:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:10 smithi169 ceph-mon[33112]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T00:29:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:10 smithi040 ceph-mon[32945]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T00:29:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:111669149759 2024-04-25T00:29:11.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T00:29:11.747 INFO:teuthology.orchestra.run.smithi031.stdout:167503724595 2024-04-25T00:29:11.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T00:29:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 -- 172.21.15.31:0/1354286025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1008f0 msgr2=0x7f498c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 --2- 172.21.15.31:0/1354286025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1008f0 0x7f498c100d70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4980009990 tx=0x7f4980009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 -- 172.21.15.31:0/1354286025 shutdown_connections 2024-04-25T00:29:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 --2- 172.21.15.31:0/1354286025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c1012b0 0x7f498c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 --2- 172.21.15.31:0/1354286025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498c1068a0 0x7f498c106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 --2- 172.21.15.31:0/1354286025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1008f0 0x7f498c100d70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 -- 172.21.15.31:0/1354286025 >> 172.21.15.31:0/1354286025 conn(0x7f498c0777a0 msgr2=0x7f498c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 -- 172.21.15.31:0/1354286025 shutdown_connections 2024-04-25T00:29:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.226+0000 7f4991de8700 1 -- 172.21.15.31:0/1354286025 wait complete. 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 Processor -- start 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 -- start start 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498c1008f0 0x7f498c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c1012b0 0x7f498c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1068a0 0x7f498c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f498c110260 con 0x7f498c1068a0 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1100e0 con 0x7f498c1012b0 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f4991de8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1103e0 con 0x7f498c1008f0 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f498bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1068a0 0x7f498c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f498bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1068a0 0x7f498c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36734/0 (socket says 172.21.15.31:36734) 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f498bfff700 1 -- 172.21.15.31:0/4145236133 learned_addr learned my addr 172.21.15.31:0/4145236133 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f498b7fe700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498c1008f0 0x7f498c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:12.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.227+0000 7f498affd700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c1012b0 0x7f498c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f498bfff700 1 -- 172.21.15.31:0/4145236133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498c1008f0 msgr2=0x7f498c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f498bfff700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498c1008f0 0x7f498c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f498bfff700 1 -- 172.21.15.31:0/4145236133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c1012b0 msgr2=0x7f498c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f498bfff700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c1012b0 0x7f498c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f498bfff700 1 -- 172.21.15.31:0/4145236133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c199080 con 0x7f498c1068a0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f498bfff700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1068a0 0x7f498c1987a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4984007ee0 tx=0x7f498400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400dd20 con 0x7f498c1068a0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4984004020 con 0x7f498c1068a0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c199370 con 0x7f498c1068a0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984015660 con 0x7f498c1068a0 2024-04-25T00:29:12.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.228+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1a5600 con 0x7f498c1068a0 2024-04-25T00:29:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.229+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f498c0629c0 con 0x7f498c1068a0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.230+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f49840041d0 con 0x7f498c1068a0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f4988ff9700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49740a2d30 0x7f49740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f498b7fe700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49740a2d30 0x7f49740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f49840cf4f0 con 0x7f498c1068a0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f4988ff9700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] conn(0x7f49740ac510 0x7f49740ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 --> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4984020e50 con 0x7f49740ac510 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f49840cf9d0 con 0x7f498c1068a0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f498affd700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] conn(0x7f49740ac510 0x7f49740ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.231+0000 7f498b7fe700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49740a2d30 0x7f49740a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f497c00bcf0 tx=0x7f497c00b4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.232+0000 7f498affd700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] conn(0x7f49740ac510 0x7f49740ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.232+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== osd.0 v2:172.21.15.31:6802/2938713990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4984020e50 con 0x7f49740ac510 2024-04-25T00:29:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 --> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f498c062e40 con 0x7f49740ac510 2024-04-25T00:29:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4988ff9700 1 -- 172.21.15.31:0/4145236133 <== osd.0 v2:172.21.15.31:6802/2938713990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f498c062e40 con 0x7f49740ac510 2024-04-25T00:29:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] conn(0x7f49740ac510 msgr2=0x7f49740ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] conn(0x7f49740ac510 0x7f49740ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49740a2d30 msgr2=0x7f49740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49740a2d30 0x7f49740a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f497c00bcf0 tx=0x7f497c00b4c0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1068a0 msgr2=0x7f498c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 --2- 172.21.15.31:0/4145236133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c1068a0 0x7f498c1987a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4984007ee0 tx=0x7f498400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f498bfff700 1 -- 172.21.15.31:0/4145236133 reap_dead start 2024-04-25T00:29:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 shutdown_connections 2024-04-25T00:29:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.249+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 >> 172.21.15.31:0/4145236133 conn(0x7f498c0777a0 msgr2=0x7f498c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.250+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 shutdown_connections 2024-04-25T00:29:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:12.250+0000 7f4991de8700 1 -- 172.21.15.31:0/4145236133 wait complete. 2024-04-25T00:29:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:12 smithi031 ceph-mon[29167]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:12 smithi169 ceph-mon[33112]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:12 smithi040 ceph-mon[32945]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:12.998 INFO:teuthology.orchestra.run.smithi031.stdout:34359738446 2024-04-25T00:29:12.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T00:29:13.490 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.492 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.492 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.495 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.538 INFO:teuthology.orchestra.run.smithi031.stdout:77309411397 2024-04-25T00:29:13.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T00:29:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.598 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-25T00:29:13.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:13 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T00:29:13.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-04-25T00:29:13.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-25T00:29:13.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:13 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T00:29:13.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-04-25T00:29:13.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-25T00:29:13.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:13 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T00:29:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T00:29:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2024-04-25T00:29:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-25T00:29:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T00:29:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T00:29:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T00:29:14.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T00:29:14.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-04-25T00:29:14.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-25T00:29:14.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T00:29:14.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:14 smithi031 ceph-mon[29167]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T00:29:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T00:29:14.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-04-25T00:29:14.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-25T00:29:14.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T00:29:14.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:14 smithi169 ceph-mon[33112]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:29:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T00:29:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2024-04-25T00:29:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-25T00:29:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T00:29:14.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:14 smithi040 ceph-mon[32945]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:29:15.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:15 smithi031 ceph-mon[29167]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:29:15.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:15 smithi169 ceph-mon[33112]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:29:15.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:15 smithi040 ceph-mon[32945]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:29:16.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:16 smithi031 ceph-mon[29167]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T00:29:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:16 smithi169 ceph-mon[33112]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T00:29:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:16 smithi040 ceph-mon[32945]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T00:29:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:18 smithi031 ceph-mon[29167]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-04-25T00:29:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:18 smithi169 ceph-mon[33112]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-04-25T00:29:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:18 smithi040 ceph-mon[32945]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-04-25T00:29:20.699 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:20 smithi031 ceph-mon[29167]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T00:29:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:20 smithi169 ceph-mon[33112]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T00:29:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:20 smithi040 ceph-mon[32945]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T00:29:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:22.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:22 smithi031 ceph-mon[29167]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T00:29:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:22 smithi169 ceph-mon[33112]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T00:29:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:22 smithi040 ceph-mon[32945]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T00:29:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 -- 172.21.15.31:0/229398015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474101af0 msgr2=0x7f647410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 --2- 172.21.15.31:0/229398015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474101af0 0x7f647410dfe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f646c00b0a0 tx=0x7f646c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 -- 172.21.15.31:0/229398015 shutdown_connections 2024-04-25T00:29:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 --2- 172.21.15.31:0/229398015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474101af0 0x7f647410dfe0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 --2- 172.21.15.31:0/229398015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64741071a0 0x7f6474107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 --2- 172.21.15.31:0/229398015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101130 0x7f64741015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 -- 172.21.15.31:0/229398015 >> 172.21.15.31:0/229398015 conn(0x7f64740780e0 msgr2=0x7f64740784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 -- 172.21.15.31:0/229398015 shutdown_connections 2024-04-25T00:29:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.159+0000 7f6478c9f700 1 -- 172.21.15.31:0/229398015 wait complete. 2024-04-25T00:29:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 Processor -- start 2024-04-25T00:29:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 -- start start 2024-04-25T00:29:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6474101130 0x7f647419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101af0 0x7f647419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 0x7f6474197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6474111330 con 0x7f64741071a0 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64741111b0 con 0x7f6474101af0 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.160+0000 7f6478c9f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64741114b0 con 0x7f6474101130 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 0x7f6474197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 0x7f6474197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40518/0 (socket says 172.21.15.31:40518) 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 -- 172.21.15.31:0/3272261331 learned_addr learned my addr 172.21.15.31:0/3272261331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6471d9b700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101af0 0x7f647419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f647259c700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6474101130 0x7f647419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 -- 172.21.15.31:0/3272261331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6474101130 msgr2=0x7f647419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6474101130 0x7f647419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 -- 172.21.15.31:0/3272261331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101af0 msgr2=0x7f647419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101af0 0x7f647419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 -- 172.21.15.31:0/3272261331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6474197d70 con 0x7f64741071a0 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6471d9b700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101af0 0x7f647419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f6472d9d700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 0x7f6474197610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f646c00b070 tx=0x7f646c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c021070 con 0x7f64741071a0 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.161+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f646c004e40 con 0x7f64741071a0 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.162+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6474198000 con 0x7f64741071a0 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.162+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c004630 con 0x7f64741071a0 2024-04-25T00:29:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.162+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6474069f10 con 0x7f64741071a0 2024-04-25T00:29:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.163+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6454005330 con 0x7f64741071a0 2024-04-25T00:29:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.164+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f646c003c00 con 0x7f64741071a0 2024-04-25T00:29:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.165+0000 7f646b7fe700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f64600a2cb0 0x7f64600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.165+0000 7f647259c700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f64600a2cb0 0x7f64600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.165+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f646c00f020 con 0x7f64741071a0 2024-04-25T00:29:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.165+0000 7f647259c700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f64600a2cb0 0x7f64600a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6464010d00 tx=0x7f6464012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:23.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.170+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f646c02a020 con 0x7f64741071a0 2024-04-25T00:29:23.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.383+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f64540058f0 con 0x7f64741071a0 2024-04-25T00:29:23.384 INFO:teuthology.orchestra.run.smithi031.stdout:261993005096 2024-04-25T00:29:23.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.384+0000 7f646b7fe700 1 -- 172.21.15.31:0/3272261331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f646c09d220 con 0x7f64741071a0 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f64600a2cb0 msgr2=0x7f64600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f64600a2cb0 0x7f64600a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6464010d00 tx=0x7f6464012040 comp rx=0 tx=0).stop 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 msgr2=0x7f6474197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 0x7f6474197610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f646c00b070 tx=0x7f646c009c90 comp rx=0 tx=0).stop 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 shutdown_connections 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f64600a2cb0 0x7f64600a5170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64741071a0 0x7f6474197610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6474101130 0x7f647419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 --2- 172.21.15.31:0/3272261331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6474101af0 0x7f647419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 >> 172.21.15.31:0/3272261331 conn(0x7f64740780e0 msgr2=0x7f64740ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 shutdown_connections 2024-04-25T00:29:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.386+0000 7f6478c9f700 1 -- 172.21.15.31:0/3272261331 wait complete. 2024-04-25T00:29:23.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:23 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:23.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:23 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3272261331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:29:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:23 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:23.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:23 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3272261331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:29:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:23 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:23.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:23 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3272261331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:29:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 -- 172.21.15.31:0/1597149521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 msgr2=0x7f12f01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1597149521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01017a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f12e00099c0 tx=0x7f12e0009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 -- 172.21.15.31:0/1597149521 shutdown_connections 2024-04-25T00:29:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1597149521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01017a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1597149521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0106450 0x7f12f010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1597149521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12f0101d70 0x7f12f0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 -- 172.21.15.31:0/1597149521 >> 172.21.15.31:0/1597149521 conn(0x7f12f00780e0 msgr2=0x7f12f00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 -- 172.21.15.31:0/1597149521 shutdown_connections 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.394+0000 7f12f5e19700 1 -- 172.21.15.31:0/1597149521 wait complete. 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 Processor -- start 2024-04-25T00:29:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 -- start start 2024-04-25T00:29:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 0x7f12f01a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12f0106450 0x7f12f019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0111330 con 0x7f12f01013c0 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01111b0 con 0x7f12f0101d70 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12f5e19700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01114b0 con 0x7f12f0106450 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12eeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 0x7f12f01a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12eeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 0x7f12f01a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:45070/0 (socket says 172.21.15.31:45070) 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12eeffd700 1 -- 172.21.15.31:0/1523998235 learned_addr learned my addr 172.21.15.31:0/1523998235 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40556/0 (socket says 172.21.15.31:40556) 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.395+0000 7f12effff700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12f0106450 0x7f12f019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12eeffd700 1 -- 172.21.15.31:0/1523998235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12f0106450 msgr2=0x7f12f019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12eeffd700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12f0106450 0x7f12f019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12eeffd700 1 -- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 msgr2=0x7f12f01a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12eeffd700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12eeffd700 1 -- 172.21.15.31:0/1523998235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f019b150 con 0x7f12f0101d70 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12ef7fe700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12eeffd700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 0x7f12f01a0d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12e400d980 tx=0x7f12e400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4017070 con 0x7f12f0101d70 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e400fd40 con 0x7f12f0101d70 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f019b440 con 0x7f12f0101d70 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e400e3b0 con 0x7f12f0101d70 2024-04-25T00:29:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.396+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f01a7700 con 0x7f12f0101d70 2024-04-25T00:29:24.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.397+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f0068f20 con 0x7f12f0101d70 2024-04-25T00:29:24.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.399+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f12e400f890 con 0x7f12f0101d70 2024-04-25T00:29:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.400+0000 7f12ecff9700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12d80a2cb0 0x7f12d80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.400+0000 7f12ef7fe700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12d80a2cb0 0x7f12d80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.400+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f12e40cec40 con 0x7f12f0101d70 2024-04-25T00:29:24.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.400+0000 7f12ef7fe700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12d80a2cb0 0x7f12d80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12e0000c00 tx=0x7f12e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.407+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12e409ba30 con 0x7f12f0101d70 2024-04-25T00:29:24.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.617+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f12f002db30 con 0x7f12f0101d70 2024-04-25T00:29:24.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.618+0000 7f12ecff9700 1 -- 172.21.15.31:0/1523998235 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f12e409b330 con 0x7f12f0101d70 2024-04-25T00:29:24.618 INFO:teuthology.orchestra.run.smithi031.stdout:201863462963 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12d80a2cb0 msgr2=0x7f12d80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12d80a2cb0 0x7f12d80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12e0000c00 tx=0x7f12e0005fb0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 msgr2=0x7f12f01a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 0x7f12f01a0d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12e400d980 tx=0x7f12e400dc90 comp rx=0 tx=0).stop 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 shutdown_connections 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12d80a2cb0 0x7f12d80a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12f01013c0 0x7f12f01a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12f0106450 0x7f12f019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 --2- 172.21.15.31:0/1523998235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f0101d70 0x7f12f01a0d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 >> 172.21.15.31:0/1523998235 conn(0x7f12f00780e0 msgr2=0x7f12f0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 shutdown_connections 2024-04-25T00:29:24.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.620+0000 7f12f5e19700 1 -- 172.21.15.31:0/1523998235 wait complete. 2024-04-25T00:29:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:24 smithi031 ceph-mon[29167]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 11 B/s, 1 objects/s recovering 2024-04-25T00:29:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:24 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/244962365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:29:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:24 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.790 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:24 smithi169 ceph-mon[33112]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 11 B/s, 1 objects/s recovering 2024-04-25T00:29:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:24 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/244962365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:29:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:24 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:24 smithi040 ceph-mon[32945]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 11 B/s, 1 objects/s recovering 2024-04-25T00:29:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:24 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/244962365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:29:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:24 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:24.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/3468768201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf0806a8e0 msgr2=0x7fcf0810f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/3468768201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf0806a8e0 0x7fcf0810f0f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000b0a0 tx=0x7fcf0000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/3468768201 shutdown_connections 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/3468768201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf0806a8e0 0x7fcf0810f0f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/3468768201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf081082b0 0x7fcf08108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/3468768201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf08069f40 0x7fcf0806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/3468768201 >> 172.21.15.31:0/3468768201 conn(0x7fcf080780e0 msgr2=0x7fcf080784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/3468768201 shutdown_connections 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.827+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/3468768201 wait complete. 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 Processor -- start 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 -- start start 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 0x7fcf0819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf0806a8e0 0x7fcf0819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf081082b0 0x7fcf08198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf081123b0 con 0x7fcf08069f40 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08112230 con 0x7fcf081082b0 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0e8e4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08112530 con 0x7fcf0806a8e0 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 0x7fcf0819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 0x7fcf0819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40538/0 (socket says 172.21.15.31:40538) 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 -- 172.21.15.31:0/244962365 learned_addr learned my addr 172.21.15.31:0/244962365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf0ce81700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf081082b0 0x7fcf08198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf077fe700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf0806a8e0 0x7fcf0819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 -- 172.21.15.31:0/244962365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf0806a8e0 msgr2=0x7fcf0819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf0806a8e0 0x7fcf0819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 -- 172.21.15.31:0/244962365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf081082b0 msgr2=0x7fcf08198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf081082b0 0x7fcf08198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.828+0000 7fcf07fff700 1 -- 172.21.15.31:0/244962365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf08199070 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.829+0000 7fcf07fff700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 0x7fcf0819e670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d950 tx=0x7fcef800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.829+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8016070 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.829+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcef8009df0 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.829+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf08199360 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.829+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef800f3c0 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.829+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf081a55f0 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.830+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf08103960 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.831+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcef8019070 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.832+0000 7fcf057fa700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcef00a2d30 0x7fcef00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.832+0000 7fcf077fe700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcef00a2d30 0x7fcef00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.832+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcef80cecf0 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.832+0000 7fcf077fe700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcef00a2d30 0x7fcef00a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005950 tx=0x7fcefc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:23.835+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcef809ba80 con 0x7fcf08069f40 2024-04-25T00:29:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.052+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcf08199c70 con 0x7fcf08069f40 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.052+0000 7fcf057fa700 1 -- 172.21.15.31:0/244962365 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcef809b380 con 0x7fcf08069f40 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stdout:146028888126 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcef00a2d30 msgr2=0x7fcef00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcef00a2d30 0x7fcef00a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005950 tx=0x7fcefc0058e0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 msgr2=0x7fcf0819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 0x7fcf0819e670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d950 tx=0x7fcef800dc60 comp rx=0 tx=0).stop 2024-04-25T00:29:24.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 shutdown_connections 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcef00a2d30 0x7fcef00a51f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf08069f40 0x7fcf0819e670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf081082b0 0x7fcf08198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 --2- 172.21.15.31:0/244962365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf0806a8e0 0x7fcf0819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 >> 172.21.15.31:0/244962365 conn(0x7fcf080780e0 msgr2=0x7fcf080ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 shutdown_connections 2024-04-25T00:29:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:24.054+0000 7fcf0e8e4700 1 -- 172.21.15.31:0/244962365 wait complete. 2024-04-25T00:29:25.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:25 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1523998235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:29:25.690 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888126 for osd.4 2024-04-25T00:29:25.690 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:25.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:25 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1523998235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:29:25.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:25 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1523998235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:29:26.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 -- 172.21.15.31:0/638046550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 msgr2=0x7f5aa01028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:26.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 --2- 172.21.15.31:0/638046550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa01028f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0099c0 tx=0x7f5a9c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 -- 172.21.15.31:0/638046550 shutdown_connections 2024-04-25T00:29:26.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 --2- 172.21.15.31:0/638046550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa01028f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 --2- 172.21.15.31:0/638046550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5aa01075e0 0x7f5aa010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 --2- 172.21.15.31:0/638046550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa0102ec0 0x7f5aa0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 -- 172.21.15.31:0/638046550 >> 172.21.15.31:0/638046550 conn(0x7f5aa00793e0 msgr2=0x7f5aa00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.613+0000 7f5aa778e700 1 -- 172.21.15.31:0/638046550 shutdown_connections 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.614+0000 7f5aa778e700 1 -- 172.21.15.31:0/638046550 wait complete. 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.614+0000 7f5aa778e700 1 Processor -- start 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.614+0000 7f5aa778e700 1 -- start start 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.614+0000 7f5aa778e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa0074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.614+0000 7f5aa778e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5aa0102ec0 0x7f5aa0074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.614+0000 7f5aa778e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa01075e0 0x7f5aa0074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa778e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0112490 con 0x7f5aa0102510 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa778e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0112310 con 0x7f5aa0102ec0 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa778e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0112610 con 0x7f5aa01075e0 2024-04-25T00:29:26.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa0074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa0074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40570/0 (socket says 172.21.15.31:40570) 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 -- 172.21.15.31:0/451299639 learned_addr learned my addr 172.21.15.31:0/451299639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa4d29700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5aa0102ec0 0x7f5aa0074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa5d2b700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa01075e0 0x7f5aa0074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 -- 172.21.15.31:0/451299639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa01075e0 msgr2=0x7f5aa0074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa01075e0 0x7f5aa0074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 -- 172.21.15.31:0/451299639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5aa0102ec0 msgr2=0x7f5aa0074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5aa0102ec0 0x7f5aa0074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 -- 172.21.15.31:0/451299639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01a6f10 con 0x7f5aa0102510 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5aa552a700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa0074530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006010 tx=0x7f5a9c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.615+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c027070 con 0x7f5aa0102510 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.616+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01a71a0 con 0x7f5aa0102510 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.616+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a9c003f50 con 0x7f5aa0102510 2024-04-25T00:29:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.616+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c022cd0 con 0x7f5aa0102510 2024-04-25T00:29:26.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.616+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a7650 con 0x7f5aa0102510 2024-04-25T00:29:26.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.617+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa0068f20 con 0x7f5aa0102510 2024-04-25T00:29:26.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.619+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5a9c02a070 con 0x7f5aa0102510 2024-04-25T00:29:26.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.619+0000 7f5a967fc700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5a8c0a2d30 0x7f5a8c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:26.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.620+0000 7f5aa4d29700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5a8c0a2d30 0x7f5a8c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:26.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.620+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f5a9c0cffe0 con 0x7f5aa0102510 2024-04-25T00:29:26.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.620+0000 7f5aa4d29700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5a8c0a2d30 0x7f5a8c0a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0101ee0 tx=0x7f5a9000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.623+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a9c013070 con 0x7f5aa0102510 2024-04-25T00:29:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:26 smithi031 ceph-mon[29167]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-04-25T00:29:26.796 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:26 smithi040 ceph-mon[32945]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-04-25T00:29:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.830+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5aa002db30 con 0x7f5aa0102510 2024-04-25T00:29:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.830+0000 7f5a967fc700 1 -- 172.21.15.31:0/451299639 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a9c09ce60 con 0x7f5aa0102510 2024-04-25T00:29:26.831 INFO:teuthology.orchestra.run.smithi031.stdout:111669149762 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5a8c0a2d30 msgr2=0x7f5a8c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5a8c0a2d30 0x7f5a8c0a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0101ee0 tx=0x7f5a9000a300 comp rx=0 tx=0).stop 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 msgr2=0x7f5aa0074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa0074530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006010 tx=0x7f5a9c009f80 comp rx=0 tx=0).stop 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 shutdown_connections 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5a8c0a2d30 0x7f5a8c0a51f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa0102510 0x7f5aa0074530 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa01075e0 0x7f5aa0074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 --2- 172.21.15.31:0/451299639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5aa0102ec0 0x7f5aa0074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:26.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 >> 172.21.15.31:0/451299639 conn(0x7f5aa00793e0 msgr2=0x7f5aa0100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:26.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.832+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 shutdown_connections 2024-04-25T00:29:26.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:26.833+0000 7f5aa778e700 1 -- 172.21.15.31:0/451299639 wait complete. 2024-04-25T00:29:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:26 smithi169 ceph-mon[33112]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-04-25T00:29:27.729 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:27 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/451299639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:29:27.816 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:27 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/451299639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:29:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.857+0000 7f55bfa1b700 1 -- 172.21.15.31:0/1998761043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 msgr2=0x7f55b8101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.857+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/1998761043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8101790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f55b40099c0 tx=0x7f55b4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 -- 172.21.15.31:0/1998761043 shutdown_connections 2024-04-25T00:29:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/1998761043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8101790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/1998761043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55b8106490 0x7f55b810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/1998761043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8101d60 0x7f55b8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 -- 172.21.15.31:0/1998761043 >> 172.21.15.31:0/1998761043 conn(0x7f55b80781d0 msgr2=0x7f55b80785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 -- 172.21.15.31:0/1998761043 shutdown_connections 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 -- 172.21.15.31:0/1998761043 wait complete. 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 Processor -- start 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.858+0000 7f55bfa1b700 1 -- start start 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55b8101d60 0x7f55b8103390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8106490 0x7f55b81038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8111320 con 0x7f55b81013b0 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81111a0 con 0x7f55b8101d60 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81114a0 con 0x7f55b8106490 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8102e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40576/0 (socket says 172.21.15.31:40576) 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 -- 172.21.15.31:0/2277710278 learned_addr learned my addr 172.21.15.31:0/2277710278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bcfb6700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55b8101d60 0x7f55b8103390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bdfb8700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8106490 0x7f55b81038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 -- 172.21.15.31:0/2277710278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8106490 msgr2=0x7f55b81038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8106490 0x7f55b81038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 -- 172.21.15.31:0/2277710278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55b8101d60 msgr2=0x7f55b8103390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55b8101d60 0x7f55b8103390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 -- 172.21.15.31:0/2277710278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81a6e30 con 0x7f55b81013b0 2024-04-25T00:29:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bdfb8700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8106490 0x7f55b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bd7b7700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8102e50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f55b40059d0 tx=0x7f55b40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4027070 con 0x7f55b81013b0 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a70c0 con 0x7f55b81013b0 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b4003f50 con 0x7f55b81013b0 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.859+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4022d80 con 0x7f55b81013b0 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.860+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a7510 con 0x7f55b81013b0 2024-04-25T00:29:27.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.861+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8068f20 con 0x7f55b81013b0 2024-04-25T00:29:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.862+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f55b402a070 con 0x7f55b81013b0 2024-04-25T00:29:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.863+0000 7f55ae7fc700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55a40a2cb0 0x7f55a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.863+0000 7f55bcfb6700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55a40a2cb0 0x7f55a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.863+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f55b40d0630 con 0x7f55b81013b0 2024-04-25T00:29:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.863+0000 7f55bcfb6700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55a40a2cb0 0x7f55a40a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f55b8105fe0 tx=0x7f55a800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:27.868+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f55b4031020 con 0x7f55b81013b0 2024-04-25T00:29:27.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:27 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/451299639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:29:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.077+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f55b802dcf0 con 0x7f55b81013b0 2024-04-25T00:29:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.078+0000 7f55ae7fc700 1 -- 172.21.15.31:0/2277710278 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55b4013070 con 0x7f55b81013b0 2024-04-25T00:29:28.078 INFO:teuthology.orchestra.run.smithi031.stdout:167503724601 2024-04-25T00:29:28.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55a40a2cb0 msgr2=0x7f55a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55a40a2cb0 0x7f55a40a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f55b8105fe0 tx=0x7f55a800b410 comp rx=0 tx=0).stop 2024-04-25T00:29:28.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 msgr2=0x7f55b8102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8102e50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f55b40059d0 tx=0x7f55b40057c0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 shutdown_connections 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55a40a2cb0 0x7f55a40a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55b81013b0 0x7f55b8102e50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b8106490 0x7f55b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 --2- 172.21.15.31:0/2277710278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55b8101d60 0x7f55b8103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 >> 172.21.15.31:0/2277710278 conn(0x7f55b80781d0 msgr2=0x7f55b8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 shutdown_connections 2024-04-25T00:29:28.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.080+0000 7f55bfa1b700 1 -- 172.21.15.31:0/2277710278 wait complete. 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- 172.21.15.31:0/1542505657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34100c60 msgr2=0x7eff341010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- 172.21.15.31:0/1542505657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34100c60 0x7eff341010e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff24009990 tx=0x7eff24009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- 172.21.15.31:0/1542505657 shutdown_connections 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- 172.21.15.31:0/1542505657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34101620 0x7eff3410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- 172.21.15.31:0/1542505657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 0x7eff341087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- 172.21.15.31:0/1542505657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34100c60 0x7eff341010e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- 172.21.15.31:0/1542505657 >> 172.21.15.31:0/1542505657 conn(0x7eff340763a0 msgr2=0x7eff340767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- 172.21.15.31:0/1542505657 shutdown_connections 2024-04-25T00:29:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- 172.21.15.31:0/1542505657 wait complete. 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 Processor -- start 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- start start 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34100c60 0x7eff3419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 0x7eff3419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 0x7eff34198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.568+0000 7eff3b5de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff341123c0 con 0x7eff34101620 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff3b5de700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff34112240 con 0x7eff34100c60 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff3b5de700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff34112540 con 0x7eff34108410 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 0x7eff3419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 0x7eff3419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40594/0 (socket says 172.21.15.31:40594) 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 -- 172.21.15.31:0/3079353615 learned_addr learned my addr 172.21.15.31:0/3079353615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff3937a700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34100c60 0x7eff3419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff39b7b700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 0x7eff34198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 -- 172.21.15.31:0/3079353615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 msgr2=0x7eff34198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 0x7eff34198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 -- 172.21.15.31:0/3079353615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34100c60 msgr2=0x7eff3419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34100c60 0x7eff3419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 -- 172.21.15.31:0/3079353615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff34198fe0 con 0x7eff34101620 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff3937a700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34100c60 0x7eff3419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff39b7b700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 0x7eff34198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff38b79700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 0x7eff3419eb70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff240057c0 tx=0x7eff240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24026070 con 0x7eff34101620 2024-04-25T00:29:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff24021410 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff34199270 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24017650 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.569+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341a55b0 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.570+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff341995d0 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.571+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff24029070 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.572+0000 7eff2a7fc700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7eff200a2d30 0x7eff200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.572+0000 7eff3937a700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7eff200a2d30 0x7eff200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.572+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7eff240d01f0 con 0x7eff34101620 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.572+0000 7eff3937a700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7eff200a2d30 0x7eff200a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff30007f50 tx=0x7eff300073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:28.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.576+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff24030080 con 0x7eff34101620 2024-04-25T00:29:28.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:28 smithi031 ceph-mon[29167]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 1 objects/s recovering 2024-04-25T00:29:28.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:28 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2277710278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:29:28.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.787+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7eff34068f20 con 0x7eff34101620 2024-04-25T00:29:28.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.787+0000 7eff2a7fc700 1 -- 172.21.15.31:0/3079353615 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff24013070 con 0x7eff34101620 2024-04-25T00:29:28.789 INFO:teuthology.orchestra.run.smithi031.stdout:55834574925 2024-04-25T00:29:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.789+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7eff200a2d30 msgr2=0x7eff200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.789+0000 7eff3b5de700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7eff200a2d30 0x7eff200a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff30007f50 tx=0x7eff300073d0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.789+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 msgr2=0x7eff3419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.789+0000 7eff3b5de700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 0x7eff3419eb70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff240057c0 tx=0x7eff240057f0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 shutdown_connections 2024-04-25T00:29:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff34108410 0x7eff34198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff34100c60 0x7eff3419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7eff200a2d30 0x7eff200a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 --2- 172.21.15.31:0/3079353615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff34101620 0x7eff3419eb70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 >> 172.21.15.31:0/3079353615 conn(0x7eff340763a0 msgr2=0x7eff340ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:28.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 shutdown_connections 2024-04-25T00:29:28.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.790+0000 7eff3b5de700 1 -- 172.21.15.31:0/3079353615 wait complete. 2024-04-25T00:29:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:28 smithi169 ceph-mon[33112]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 1 objects/s recovering 2024-04-25T00:29:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:28 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2277710278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:29:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:28 smithi040 ceph-mon[32945]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 1 objects/s recovering 2024-04-25T00:29:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:28 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2277710278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:29:28.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 -- 172.21.15.31:0/1232024240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101130 msgr2=0x7efda41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 --2- 172.21.15.31:0/1232024240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101130 0x7efda41015b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd98009990 tx=0x7efd98009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 -- 172.21.15.31:0/1232024240 shutdown_connections 2024-04-25T00:29:28.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 --2- 172.21.15.31:0/1232024240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda4101af0 0x7efda410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 --2- 172.21.15.31:0/1232024240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda41071a0 0x7efda4107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 --2- 172.21.15.31:0/1232024240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101130 0x7efda41015b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 -- 172.21.15.31:0/1232024240 >> 172.21.15.31:0/1232024240 conn(0x7efda40780e0 msgr2=0x7efda40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.916+0000 7efda9d70700 1 -- 172.21.15.31:0/1232024240 shutdown_connections 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 -- 172.21.15.31:0/1232024240 wait complete. 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 Processor -- start 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 -- start start 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4101130 0x7efda419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 0x7efda419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda41071a0 0x7efda4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.917+0000 7efda9d70700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efda41112f0 con 0x7efda4101af0 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda9d70700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efda4111170 con 0x7efda4101130 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda9d70700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efda4111470 con 0x7efda41071a0 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 0x7efda419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4101130 0x7efda419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 0x7efda419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40612/0 (socket says 172.21.15.31:40612) 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 -- 172.21.15.31:0/3818481029 learned_addr learned my addr 172.21.15.31:0/3818481029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda3fff700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda41071a0 0x7efda4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 -- 172.21.15.31:0/3818481029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda41071a0 msgr2=0x7efda4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda41071a0 0x7efda4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 -- 172.21.15.31:0/3818481029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4101130 msgr2=0x7efda419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4101130 0x7efda419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 -- 172.21.15.31:0/3818481029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4199000 con 0x7efda4101af0 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda37fe700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4101130 0x7efda419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda2ffd700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 0x7efda419eb40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efd9800b2e0 tx=0x7efd980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98026070 con 0x7efda4101af0 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd98021410 con 0x7efda4101af0 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98017650 con 0x7efda4101af0 2024-04-25T00:29:28.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda4199290 con 0x7efda4101af0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.918+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a5580 con 0x7efda4101af0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.919+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda41995f0 con 0x7efda4101af0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.921+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efd98029070 con 0x7efda4101af0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.922+0000 7efda0ff9700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd8c0a2d30 0x7efd8c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.922+0000 7efda37fe700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd8c0a2d30 0x7efd8c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.922+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7efd980d06d0 con 0x7efda4101af0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.922+0000 7efda37fe700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd8c0a2d30 0x7efd8c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efd94009fd0 tx=0x7efd94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:28.925+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd98030070 con 0x7efda4101af0 2024-04-25T00:29:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.136+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7efda4199c00 con 0x7efda4101af0 2024-04-25T00:29:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.136+0000 7efda0ff9700 1 -- 172.21.15.31:0/3818481029 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7efd98013070 con 0x7efda4101af0 2024-04-25T00:29:29.137 INFO:teuthology.orchestra.run.smithi031.stdout:317827579935 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd8c0a2d30 msgr2=0x7efd8c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd8c0a2d30 0x7efd8c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efd94009fd0 tx=0x7efd94009260 comp rx=0 tx=0).stop 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 msgr2=0x7efda419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 0x7efda419eb40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efd9800b2e0 tx=0x7efd980047b0 comp rx=0 tx=0).stop 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 shutdown_connections 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda41071a0 0x7efda4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4101130 0x7efda419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efd8c0a2d30 0x7efd8c0a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 --2- 172.21.15.31:0/3818481029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4101af0 0x7efda419eb40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 >> 172.21.15.31:0/3818481029 conn(0x7efda40780e0 msgr2=0x7efda406a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 shutdown_connections 2024-04-25T00:29:29.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:29.138+0000 7efda9d70700 1 -- 172.21.15.31:0/3818481029 wait complete. 2024-04-25T00:29:29.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:29 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3079353615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:29:29.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:29 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3818481029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:29:29.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:29 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3079353615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:29:29.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:29 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3818481029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:29:29.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:29 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3079353615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:29:29.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:29 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3818481029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:29:30.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:30 smithi031 ceph-mon[29167]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 9 B/s, 1 objects/s recovering 2024-04-25T00:29:30.694 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462963 for osd.6 2024-04-25T00:29:30.694 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:30 smithi169 ceph-mon[33112]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 9 B/s, 1 objects/s recovering 2024-04-25T00:29:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:30 smithi040 ceph-mon[32945]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 9 B/s, 1 objects/s recovering 2024-04-25T00:29:31.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574925 for osd.1 2024-04-25T00:29:31.244 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2798146464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 msgr2=0x7f4ad010f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2798146464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad010f140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b0a0 tx=0x7f4ac800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2798146464 shutdown_connections 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2798146464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad010f140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2798146464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad01024e0 0x7f4ad01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2798146464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad0102e90 0x7f4ad0106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2798146464 >> 172.21.15.31:0/2798146464 conn(0x7f4ad0079390 msgr2=0x7f4ad00797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.097+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2798146464 shutdown_connections 2024-04-25T00:29:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.098+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2798146464 wait complete. 2024-04-25T00:29:32.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.098+0000 7f4ad7d50700 1 Processor -- start 2024-04-25T00:29:32.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.098+0000 7f4ad7d50700 1 -- start start 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad01024e0 0x7f4ad019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad0102e90 0x7f4ad019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112480 con 0x7f4ad0107570 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112300 con 0x7f4ad01024e0 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112600 con 0x7f4ad0102e90 2024-04-25T00:29:32.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40630/0 (socket says 172.21.15.31:40630) 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 -- 172.21.15.31:0/2556312327 learned_addr learned my addr 172.21.15.31:0/2556312327 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad5aec700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad01024e0 0x7f4ad019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad52eb700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad0102e90 0x7f4ad019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 -- 172.21.15.31:0/2556312327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad0102e90 msgr2=0x7f4ad019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad0102e90 0x7f4ad019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 -- 172.21.15.31:0/2556312327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad01024e0 msgr2=0x7f4ad019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad01024e0 0x7f4ad019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 -- 172.21.15.31:0/2556312327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad0199040 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad62ed700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad0198760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b830 tx=0x7f4ac8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8021070 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01992d0 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.099+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ac8007e30 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.100+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80047d0 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.100+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a5610 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.101+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad0103970 con 0x7f4ad0107570 2024-04-25T00:29:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.102+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4ac8007980 con 0x7f4ad0107570 2024-04-25T00:29:32.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.102+0000 7f4ac6ffd700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4abc0a2cb0 0x7f4abc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:32.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.102+0000 7f4ad5aec700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4abc0a2cb0 0x7f4abc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:32.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.102+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f4ac800f020 con 0x7f4ad0107570 2024-04-25T00:29:32.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.103+0000 7f4ad5aec700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4abc0a2cb0 0x7f4abc0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4acc004630 tx=0x7f4acc00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.107+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ac802a020 con 0x7f4ad0107570 2024-04-25T00:29:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.317+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4ad002db30 con 0x7f4ad0107570 2024-04-25T00:29:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.318+0000 7f4ac6ffd700 1 -- 172.21.15.31:0/2556312327 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ac8004000 con 0x7f4ad0107570 2024-04-25T00:29:32.318 INFO:teuthology.orchestra.run.smithi031.stdout:296352743460 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4abc0a2cb0 msgr2=0x7f4abc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4abc0a2cb0 0x7f4abc0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4acc004630 tx=0x7f4acc00b3c0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 msgr2=0x7f4ad0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad0198760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b830 tx=0x7f4ac8004ce0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 shutdown_connections 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4abc0a2cb0 0x7f4abc0a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad0107570 0x7f4ad0198760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad01024e0 0x7f4ad019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 --2- 172.21.15.31:0/2556312327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad0102e90 0x7f4ad019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 >> 172.21.15.31:0/2556312327 conn(0x7f4ad0079390 msgr2=0x7f4ad01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 shutdown_connections 2024-04-25T00:29:32.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.320+0000 7f4ad7d50700 1 -- 172.21.15.31:0/2556312327 wait complete. 2024-04-25T00:29:32.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:32 smithi031 ceph-mon[29167]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:32.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:32 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2556312327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:29:32.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:32 smithi169 ceph-mon[33112]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:32.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:32 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2556312327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:29:32.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:32 smithi040 ceph-mon[32945]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:32 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2556312327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:29:33.220 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005092 got 261993005096 for osd.8 2024-04-25T00:29:33.220 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:33.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 -- 172.21.15.31:0/2367704906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b81064c0 msgr2=0x7f53b810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:33.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 --2- 172.21.15.31:0/2367704906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b81064c0 0x7f53b810e060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f53b000b0d0 tx=0x7f53b000b3e0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 -- 172.21.15.31:0/2367704906 shutdown_connections 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 --2- 172.21.15.31:0/2367704906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b81064c0 0x7f53b810e060 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 --2- 172.21.15.31:0/2367704906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b8101770 0x7f53b8101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 --2- 172.21.15.31:0/2367704906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b8102120 0x7f53b8105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 -- 172.21.15.31:0/2367704906 >> 172.21.15.31:0/2367704906 conn(0x7f53b80780e0 msgr2=0x7f53b80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 -- 172.21.15.31:0/2367704906 shutdown_connections 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.433+0000 7f53c0102700 1 -- 172.21.15.31:0/2367704906 wait complete. 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 Processor -- start 2024-04-25T00:29:33.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 -- start start 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b8101770 0x7f53b819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 0x7f53b819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b81064c0 0x7f53b8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f53b8111350 con 0x7f53b8102120 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53b81111d0 con 0x7f53b81064c0 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53c0102700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53b81114d0 con 0x7f53b8101770 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53bd69d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 0x7f53b819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53bd69d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 0x7f53b819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52222/0 (socket says 172.21.15.31:52222) 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53bd69d700 1 -- 172.21.15.31:0/2205059897 learned_addr learned my addr 172.21.15.31:0/2205059897 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.434+0000 7f53be69f700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b81064c0 0x7f53b8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bde9e700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b8101770 0x7f53b819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bd69d700 1 -- 172.21.15.31:0/2205059897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b8101770 msgr2=0x7f53b819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:33.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bd69d700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b8101770 0x7f53b819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bd69d700 1 -- 172.21.15.31:0/2205059897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b81064c0 msgr2=0x7f53b8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bd69d700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b81064c0 0x7f53b8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bd69d700 1 -- 172.21.15.31:0/2205059897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b8199000 con 0x7f53b8102120 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53bd69d700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 0x7f53b819eb90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f53a800e940 tx=0x7f53a800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a800cbd0 con 0x7f53b8102120 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b81992f0 con 0x7f53b8102120 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53a800ee90 con 0x7f53b8102120 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8018680 con 0x7f53b8102120 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.435+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b81a55d0 con 0x7f53b8102120 2024-04-25T00:29:33.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.436+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b806a1b0 con 0x7f53b8102120 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.437+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f53a8010430 con 0x7f53b8102120 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.438+0000 7f53aeffd700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53a40a2cb0 0x7f53a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.438+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f53a80cf710 con 0x7f53b8102120 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.438+0000 7f53bde9e700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53a40a2cb0 0x7f53a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.438+0000 7f53bde9e700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53a40a2cb0 0x7f53a40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53b4005950 tx=0x7f53b40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:33.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.442+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53a809c500 con 0x7f53b8102120 2024-04-25T00:29:33.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149762 for osd.3 2024-04-25T00:29:33.608 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.662+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f53b8199c50 con 0x7f53b8102120 2024-04-25T00:29:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.662+0000 7f53aeffd700 1 -- 172.21.15.31:0/2205059897 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f53a809be00 con 0x7f53b8102120 2024-04-25T00:29:33.663 INFO:teuthology.orchestra.run.smithi031.stdout:352187318297 2024-04-25T00:29:33.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53a40a2cb0 msgr2=0x7f53a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53a40a2cb0 0x7f53a40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53b4005950 tx=0x7f53b40058e0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 msgr2=0x7f53b819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 0x7f53b819eb90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f53a800e940 tx=0x7f53a800ec50 comp rx=0 tx=0).stop 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 shutdown_connections 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b81064c0 0x7f53b8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b8101770 0x7f53b819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53a40a2cb0 0x7f53a40a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 --2- 172.21.15.31:0/2205059897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b8102120 0x7f53b819eb90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 >> 172.21.15.31:0/2205059897 conn(0x7f53b80780e0 msgr2=0x7f53b806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 shutdown_connections 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:33.665+0000 7f53c0102700 1 -- 172.21.15.31:0/2205059897 wait complete. 2024-04-25T00:29:33.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:33 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2343377908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:29:33.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:33 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2343377908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:29:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:33 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2343377908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:29:34.024 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:34.024 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:34.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579930 got 317827579935 for osd.10 2024-04-25T00:29:34.050 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:34 smithi031 ceph-mon[29167]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2205059897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:29:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:34 smithi169 ceph-mon[33112]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2205059897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:29:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:34 smithi040 ceph-mon[32945]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2205059897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:29:35.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318291 got 352187318297 for osd.11 2024-04-25T00:29:35.191 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:35.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 -- 172.21.15.31:0/16171077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc041071a0 msgr2=0x7efc04107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:35.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/16171077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc041071a0 0x7efc04107580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b0a0 tx=0x7efbf400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 -- 172.21.15.31:0/16171077 shutdown_connections 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/16171077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc041071a0 0x7efc04107580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/16171077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc04101af0 0x7efc0410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/16171077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc04101130 0x7efc041015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 -- 172.21.15.31:0/16171077 >> 172.21.15.31:0/16171077 conn(0x7efc040780e0 msgr2=0x7efc040784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 -- 172.21.15.31:0/16171077 shutdown_connections 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 -- 172.21.15.31:0/16171077 wait complete. 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.638+0000 7efc0bf0e700 1 Processor -- start 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 -- start start 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 0x7efc04102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc04101af0 0x7efc04103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc041071a0 0x7efc04103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc041112c0 con 0x7efc04101130 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111140 con 0x7efc04101af0 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0bf0e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111440 con 0x7efc041071a0 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 0x7efc04102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 0x7efc04102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52206/0 (socket says 172.21.15.31:52206) 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 -- 172.21.15.31:0/2343377908 learned_addr learned my addr 172.21.15.31:0/2343377908 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc0a4ab700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc041071a0 0x7efc04103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 -- 172.21.15.31:0/2343377908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc041071a0 msgr2=0x7efc04103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc041071a0 0x7efc04103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 -- 172.21.15.31:0/2343377908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc04101af0 msgr2=0x7efc04103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc094a9700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc04101af0 0x7efc04103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:35.274 INFO:teuthology.orchestra.run.smithi031.stdout:227633266736 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc04101af0 0x7efc04103350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 -- 172.21.15.31:0/2343377908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc041a6e30 con 0x7efc04101130 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.639+0000 7efc09caa700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 0x7efc04102e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b070 tx=0x7efbf4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.640+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4021070 con 0x7efc04101130 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.640+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc041a70c0 con 0x7efc04101130 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.640+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbf4007e30 con 0x7efc04101130 2024-04-25T00:29:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.640+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4004840 con 0x7efc04101130 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.640+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a75c0 con 0x7efc04101130 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.641+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc04068f20 con 0x7efc04101130 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.642+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efbf4003c00 con 0x7efc04101130 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.643+0000 7efc02ffd700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efbec0a2d30 0x7efbec0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.643+0000 7efc094a9700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efbec0a2d30 0x7efbec0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.643+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7efbf400f020 con 0x7efc04101130 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.643+0000 7efc094a9700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efbec0a2d30 0x7efbec0a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc04078650 tx=0x7efbf80094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.647+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efbf402a020 con 0x7efc04101130 2024-04-25T00:29:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.857+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efc041a7790 con 0x7efc04101130 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.858+0000 7efc02ffd700 1 -- 172.21.15.31:0/2343377908 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efbf409d520 con 0x7efc04101130 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efbec0a2d30 msgr2=0x7efbec0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efbec0a2d30 0x7efbec0a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc04078650 tx=0x7efbf80094f0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 msgr2=0x7efc04102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 0x7efc04102e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b070 tx=0x7efbf4004d00 comp rx=0 tx=0).stop 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 shutdown_connections 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efbec0a2d30 0x7efbec0a51f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc04101130 0x7efc04102e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc041071a0 0x7efc04103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 --2- 172.21.15.31:0/2343377908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc04101af0 0x7efc04103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 >> 172.21.15.31:0/2343377908 conn(0x7efc040780e0 msgr2=0x7efc0406a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 shutdown_connections 2024-04-25T00:29:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:32.860+0000 7efc0bf0e700 1 -- 172.21.15.31:0/2343377908 wait complete. 2024-04-25T00:29:36.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266729 got 227633266736 for osd.7 2024-04-25T00:29:36.020 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:36.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:36 smithi031 ceph-mon[29167]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:36.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724595 got 167503724601 for osd.5 2024-04-25T00:29:36.674 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:36 smithi169 ceph-mon[33112]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:36 smithi040 ceph-mon[32945]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:38.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:38 smithi031 ceph-mon[29167]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:29:38.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743454 got 296352743460 for osd.9 2024-04-25T00:29:38.538 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:38 smithi169 ceph-mon[33112]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:29:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:38 smithi040 ceph-mon[32945]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:29:39.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 -- 172.21.15.31:0/559064786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528101ae0 msgr2=0x7f452810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 --2- 172.21.15.31:0/559064786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528101ae0 0x7f452810dfd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f452000b0a0 tx=0x7f452000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 -- 172.21.15.31:0/559064786 shutdown_connections 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 --2- 172.21.15.31:0/559064786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528101ae0 0x7f452810dfd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 --2- 172.21.15.31:0/559064786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528107190 0x7f4528107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 --2- 172.21.15.31:0/559064786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4528101120 0x7f45281015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 -- 172.21.15.31:0/559064786 >> 172.21.15.31:0/559064786 conn(0x7f45280780a0 msgr2=0x7f45280784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 -- 172.21.15.31:0/559064786 shutdown_connections 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.530+0000 7f452ef3a700 1 -- 172.21.15.31:0/559064786 wait complete. 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 Processor -- start 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 -- start start 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4528101120 0x7f45281166b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528101ae0 0x7f4528116bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 0x7f45281117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45281101a0 con 0x7f4528107190 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4528110020 con 0x7f4528101120 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ef3a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4528110320 con 0x7f4528101ae0 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 0x7f45281117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 0x7f45281117d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52246/0 (socket says 172.21.15.31:52246) 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 -- 172.21.15.31:0/472006964 learned_addr learned my addr 172.21.15.31:0/472006964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452ccd6700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4528101120 0x7f45281166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f4527fff700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528101ae0 0x7f4528116bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 -- 172.21.15.31:0/472006964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528101ae0 msgr2=0x7f4528116bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528101ae0 0x7f4528116bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 -- 172.21.15.31:0/472006964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4528101120 msgr2=0x7f45281166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4528101120 0x7f45281166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.531+0000 7f452d4d7700 1 -- 172.21.15.31:0/472006964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45281120d0 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.532+0000 7f452d4d7700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 0x7f45281117d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f45200062a0 tx=0x7f4520004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.532+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520021070 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.532+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4520007e30 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.532+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4528112360 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.532+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200046d0 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.532+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452811a590 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.533+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4528102840 con 0x7f4528107190 2024-04-25T00:29:39.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.535+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4520007980 con 0x7f4528107190 2024-04-25T00:29:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.536+0000 7f4525ffb700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45180a2c90 0x7f45180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.536+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f452000f020 con 0x7f4528107190 2024-04-25T00:29:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.536+0000 7f452ccd6700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45180a2c90 0x7f45180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.536+0000 7f452ccd6700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45180a2c90 0x7f45180a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f451c007b60 tx=0x7f451c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.539+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f452002a020 con 0x7f4528107190 2024-04-25T00:29:39.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.748+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4528112d00 con 0x7f4528107190 2024-04-25T00:29:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.748+0000 7f4525ffb700 1 -- 172.21.15.31:0/472006964 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f452009d880 con 0x7f4528107190 2024-04-25T00:29:39.749 INFO:teuthology.orchestra.run.smithi031.stdout:34359738452 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45180a2c90 msgr2=0x7f45180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45180a2c90 0x7f45180a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f451c007b60 tx=0x7f451c0073d0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 msgr2=0x7f45281117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 0x7f45281117d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f45200062a0 tx=0x7f4520004c80 comp rx=0 tx=0).stop 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 shutdown_connections 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45180a2c90 0x7f45180a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4528107190 0x7f45281117d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4528101120 0x7f45281166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 --2- 172.21.15.31:0/472006964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4528101ae0 0x7f4528116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 >> 172.21.15.31:0/472006964 conn(0x7f45280780a0 msgr2=0x7f45280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 shutdown_connections 2024-04-25T00:29:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:39.750+0000 7f452ef3a700 1 -- 172.21.15.31:0/472006964 wait complete. 2024-04-25T00:29:40.477 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738452 for osd.0 2024-04-25T00:29:40.477 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:40.702 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:40 smithi031 ceph-mon[29167]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T00:29:40.702 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:40 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/472006964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:29:40.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:40 smithi169 ceph-mon[33112]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T00:29:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:40 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/472006964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:29:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:40 smithi040 ceph-mon[32945]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T00:29:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:40 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/472006964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 -- 172.21.15.31:0/2147072462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f78069f40 msgr2=0x7f8f7806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/2147072462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f78069f40 0x7f8f7806a3a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009990 tx=0x7f8f6c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 -- 172.21.15.31:0/2147072462 shutdown_connections 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/2147072462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7806a8e0 0x7f8f7810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/2147072462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781082b0 0x7f8f78108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/2147072462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f78069f40 0x7f8f7806a3a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 -- 172.21.15.31:0/2147072462 >> 172.21.15.31:0/2147072462 conn(0x7f8f780780e0 msgr2=0x7f8f780784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 -- 172.21.15.31:0/2147072462 shutdown_connections 2024-04-25T00:29:41.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.069+0000 7f8f7ff01700 1 -- 172.21.15.31:0/2147072462 wait complete. 2024-04-25T00:29:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 Processor -- start 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 -- start start 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f78069f40 0x7f8f7819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 0x7f8f7819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f781082b0 0x7f8f781987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112420 con 0x7f8f7806a8e0 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781122a0 con 0x7f8f781082b0 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7ff01700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781125a0 con 0x7f8f78069f40 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7d49c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 0x7f8f7819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7d49c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 0x7f8f7819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52278/0 (socket says 172.21.15.31:52278) 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7d49c700 1 -- 172.21.15.31:0/244672888 learned_addr learned my addr 172.21.15.31:0/244672888 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7e49e700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f781082b0 0x7f8f781987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7dc9d700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f78069f40 0x7f8f7819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.070+0000 7f8f7d49c700 1 -- 172.21.15.31:0/244672888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f78069f40 msgr2=0x7f8f7819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7d49c700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f78069f40 0x7f8f7819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7d49c700 1 -- 172.21.15.31:0/244672888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f781082b0 msgr2=0x7f8f781987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7d49c700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f781082b0 0x7f8f781987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7d49c700 1 -- 172.21.15.31:0/244672888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f78198fa0 con 0x7f8f7806a8e0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7d49c700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 0x7f8f7819eb80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c000c00 tx=0x7f8f6c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c027070 con 0x7f8f7806a8e0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c00bdc0 con 0x7f8f7806a8e0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f78199230 con 0x7f8f7806a8e0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c022d60 con 0x7f8f7806a8e0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.071+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f781a54a0 con 0x7f8f7806a8e0 2024-04-25T00:29:41.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.072+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7810a820 con 0x7f8f7806a8e0 2024-04-25T00:29:41.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.074+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8f6c02a070 con 0x7f8f7806a8e0 2024-04-25T00:29:41.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.075+0000 7f8f76ffd700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f600a2cb0 0x7f8f600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:41.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.075+0000 7f8f7dc9d700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f600a2cb0 0x7f8f600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:41.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.075+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f8f6c0d0740 con 0x7f8f7806a8e0 2024-04-25T00:29:41.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.075+0000 7f8f7dc9d700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f600a2cb0 0x7f8f600a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009f20 tx=0x7f8f700094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.078+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f6c031070 con 0x7f8f7806a8e0 2024-04-25T00:29:41.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.299+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8f78199cf0 con 0x7f8f7806a8e0 2024-04-25T00:29:41.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.299+0000 7f8f76ffd700 1 -- 172.21.15.31:0/244672888 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f6c013070 con 0x7f8f7806a8e0 2024-04-25T00:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout:77309411403 2024-04-25T00:29:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f600a2cb0 msgr2=0x7f8f600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f600a2cb0 0x7f8f600a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009f20 tx=0x7f8f700094f0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 msgr2=0x7f8f7819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 0x7f8f7819eb80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c000c00 tx=0x7f8f6c003a80 comp rx=0 tx=0).stop 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 shutdown_connections 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f781082b0 0x7f8f781987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f78069f40 0x7f8f7819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8f600a2cb0 0x7f8f600a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 --2- 172.21.15.31:0/244672888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7806a8e0 0x7f8f7819eb80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.301+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 >> 172.21.15.31:0/244672888 conn(0x7f8f780780e0 msgr2=0x7f8f780ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.302+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 shutdown_connections 2024-04-25T00:29:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:41.302+0000 7f8f7ff01700 1 -- 172.21.15.31:0/244672888 wait complete. 2024-04-25T00:29:41.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:41 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/244672888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:29:41.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:41 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/244672888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:29:41.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:41 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/244672888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:29:42.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411403 for osd.2 2024-04-25T00:29:42.139 DEBUG:teuthology.parallel:result is None 2024-04-25T00:29:42.139 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T00:29:42.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:29:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:42.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:42 smithi031 ceph-mon[29167]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:42 smithi169 ceph-mon[33112]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:42 smithi040 ceph-mon[32945]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:43.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:43.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:43.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:43.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:43.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:43 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:43 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:43 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:29:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 -- 172.21.15.31:0/3656553098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc158104ca0 msgr2=0x7fc158105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 --2- 172.21.15.31:0/3656553098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc158104ca0 0x7fc158105100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc15000b0a0 tx=0x7fc15000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 -- 172.21.15.31:0/3656553098 shutdown_connections 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 --2- 172.21.15.31:0/3656553098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc158104ca0 0x7fc158105100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 --2- 172.21.15.31:0/3656553098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc158107190 0x7fc15806a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 --2- 172.21.15.31:0/3656553098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15806af20 0x7fc15806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 -- 172.21.15.31:0/3656553098 >> 172.21.15.31:0/3656553098 conn(0x7fc1580781d0 msgr2=0x7fc1580785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 -- 172.21.15.31:0/3656553098 shutdown_connections 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.216+0000 7fc160547700 1 -- 172.21.15.31:0/3656553098 wait complete. 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 Processor -- start 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 -- start start 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 0x7fc15819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc158104ca0 0x7fc15819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc158107190 0x7fc1581a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:44.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc158118a80 con 0x7fc15806af20 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc158118900 con 0x7fc158104ca0 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc160547700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc158118c00 con 0x7fc158107190 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 0x7fc15819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 0x7fc15819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55510/0 (socket says 172.21.15.31:55510) 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 -- 172.21.15.31:0/914566822 learned_addr learned my addr 172.21.15.31:0/914566822 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15eae4700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc158107190 0x7fc1581a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15dae2700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc158104ca0 0x7fc15819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 -- 172.21.15.31:0/914566822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc158107190 msgr2=0x7fc1581a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc158107190 0x7fc1581a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 -- 172.21.15.31:0/914566822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc158104ca0 msgr2=0x7fc15819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc158104ca0 0x7fc15819f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.217+0000 7fc15e2e3700 1 -- 172.21.15.31:0/914566822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1581a3ee0 con 0x7fc15806af20 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.218+0000 7fc15e2e3700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 0x7fc15819eed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b530 tx=0x7fc15400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.218+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154015020 con 0x7fc15806af20 2024-04-25T00:29:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.218+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc154011ce0 con 0x7fc15806af20 2024-04-25T00:29:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.218+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1581a41d0 con 0x7fc15806af20 2024-04-25T00:29:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.218+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154010dd0 con 0x7fc15806af20 2024-04-25T00:29:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.218+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1581aba50 con 0x7fc15806af20 2024-04-25T00:29:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.219+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc158068f20 con 0x7fc15806af20 2024-04-25T00:29:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.220+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc154018020 con 0x7fc15806af20 2024-04-25T00:29:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.221+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1440a2c20 0x7fc1440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.221+0000 7fc15dae2700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1440a2c20 0x7fc1440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.221+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fc1540ced80 con 0x7fc15806af20 2024-04-25T00:29:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.221+0000 7fc15dae2700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1440a2c20 0x7fc1440a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc158078820 tx=0x7fc14800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.224+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc15401f020 con 0x7fc15806af20 2024-04-25T00:29:44.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.423+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc158104390 con 0x7fc1440a2c20 2024-04-25T00:29:44.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.431+0000 7fc14f7fe700 1 -- 172.21.15.31:0/914566822 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259022 (secure 0 0 0) 0x7fc158104390 con 0x7fc1440a2c20 2024-04-25T00:29:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1440a2c20 msgr2=0x7fc1440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1440a2c20 0x7fc1440a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc158078820 tx=0x7fc14800a300 comp rx=0 tx=0).stop 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 msgr2=0x7fc15819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 0x7fc15819eed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b530 tx=0x7fc15400b840 comp rx=0 tx=0).stop 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 shutdown_connections 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1440a2c20 0x7fc1440a50e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15806af20 0x7fc15819eed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc158107190 0x7fc1581a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 --2- 172.21.15.31:0/914566822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc158104ca0 0x7fc15819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:44.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 >> 172.21.15.31:0/914566822 conn(0x7fc1580781d0 msgr2=0x7fc158103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:44.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 shutdown_connections 2024-04-25T00:29:44.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:44.434+0000 7fc160547700 1 -- 172.21.15.31:0/914566822 wait complete. 2024-04-25T00:29:44.440 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T00:29:45.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:44 smithi169 ceph-mon[33112]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:44 smithi169 ceph-mon[33112]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:29:45.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:44 smithi040 ceph-mon[32945]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:45.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:44 smithi040 ceph-mon[32945]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:29:45.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:44 smithi031 ceph-mon[29167]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:45.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:44 smithi031 ceph-mon[29167]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:29:45.203 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-04-25T00:29:43.322416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657438,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":898,"num_read_kb":568,"num_write":420,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365328,"kb_used_data":11952,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576496,"statfs":{"total":1151940427776,"available":1151566331904,"internally_reserved":0,"allocated":12238848,"data_stored":7542030,"data_compressed":177228,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003590"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.436924+0000","last_change":"2024-04-25T00:28:24.187326+0000","last_active":"2024-04-25T00:29:14.436924+0000","last_peered":"2024-04-25T00:29:14.436924+0000","last_clean":"2024-04-25T00:29:14.436924+0000","last_became_active":"2024-04-25T00:28:24.186536+0000","last_became_peered":"2024-04-25T00:28:24.186536+0000","last_unstale":"2024-04-25T00:29:14.436924+0000","last_undegraded":"2024-04-25T00:29:14.436924+0000","last_fullsized":"2024-04-25T00:29:14.436924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:25.035523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435544+0000","last_change":"2024-04-25T00:28:24.181763+0000","last_active":"2024-04-25T00:29:14.435544+0000","last_peered":"2024-04-25T00:29:14.435544+0000","last_clean":"2024-04-25T00:29:14.435544+0000","last_became_active":"2024-04-25T00:28:24.181437+0000","last_became_peered":"2024-04-25T00:28:24.181437+0000","last_unstale":"2024-04-25T00:29:14.435544+0000","last_undegraded":"2024-04-25T00:29:14.435544+0000","last_fullsized":"2024-04-25T00:29:14.435544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:54.019249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435253+0000","last_change":"2024-04-25T00:28:24.186648+0000","last_active":"2024-04-25T00:29:14.435253+0000","last_peered":"2024-04-25T00:29:14.435253+0000","last_clean":"2024-04-25T00:29:14.435253+0000","last_became_active":"2024-04-25T00:28:24.186303+0000","last_became_peered":"2024-04-25T00:28:24.186303+0000","last_unstale":"2024-04-25T00:29:14.435253+0000","last_undegraded":"2024-04-25T00:29:14.435253+0000","last_fullsized":"2024-04-25T00:29:14.435253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:14.521970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.628136+0000","last_change":"2024-04-25T00:28:24.179138+0000","last_active":"2024-04-25T00:29:14.628136+0000","last_peered":"2024-04-25T00:29:14.628136+0000","last_clean":"2024-04-25T00:29:14.628136+0000","last_became_active":"2024-04-25T00:28:24.178664+0000","last_became_peered":"2024-04-25T00:28:24.178664+0000","last_unstale":"2024-04-25T00:29:14.628136+0000","last_undegraded":"2024-04-25T00:29:14.628136+0000","last_fullsized":"2024-04-25T00:29:14.628136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:07.869662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433375+0000","last_change":"2024-04-25T00:28:24.174081+0000","last_active":"2024-04-25T00:29:14.433375+0000","last_peered":"2024-04-25T00:29:14.433375+0000","last_clean":"2024-04-25T00:29:14.433375+0000","last_became_active":"2024-04-25T00:28:24.173670+0000","last_became_peered":"2024-04-25T00:28:24.173670+0000","last_unstale":"2024-04-25T00:29:14.433375+0000","last_undegraded":"2024-04-25T00:29:14.433375+0000","last_fullsized":"2024-04-25T00:29:14.433375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:43.255598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431088+0000","last_change":"2024-04-25T00:28:24.174664+0000","last_active":"2024-04-25T00:29:14.431088+0000","last_peered":"2024-04-25T00:29:14.431088+0000","last_clean":"2024-04-25T00:29:14.431088+0000","last_became_active":"2024-04-25T00:28:24.174238+0000","last_became_peered":"2024-04-25T00:28:24.174238+0000","last_unstale":"2024-04-25T00:29:14.431088+0000","last_undegraded":"2024-04-25T00:29:14.431088+0000","last_fullsized":"2024-04-25T00:29:14.431088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:21.214976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664197+0000","last_change":"2024-04-25T00:28:24.181123+0000","last_active":"2024-04-25T00:29:14.664197+0000","last_peered":"2024-04-25T00:29:14.664197+0000","last_clean":"2024-04-25T00:29:14.664197+0000","last_became_active":"2024-04-25T00:28:24.179516+0000","last_became_peered":"2024-04-25T00:28:24.179516+0000","last_unstale":"2024-04-25T00:29:14.664197+0000","last_undegraded":"2024-04-25T00:29:14.664197+0000","last_fullsized":"2024-04-25T00:29:14.664197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:11.822968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.437326+0000","last_change":"2024-04-25T00:29:14.436445+0000","last_active":"2024-04-25T00:29:14.437326+0000","last_peered":"2024-04-25T00:29:14.437326+0000","last_clean":"2024-04-25T00:29:14.437326+0000","last_became_active":"2024-04-25T00:29:14.436069+0000","last_became_peered":"2024-04-25T00:29:14.436069+0000","last_unstale":"2024-04-25T00:29:14.437326+0000","last_undegraded":"2024-04-25T00:29:14.437326+0000","last_fullsized":"2024-04-25T00:29:14.437326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:43.030641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664707+0000","last_change":"2024-04-25T00:28:24.170818+0000","last_active":"2024-04-25T00:29:14.664707+0000","last_peered":"2024-04-25T00:29:14.664707+0000","last_clean":"2024-04-25T00:29:14.664707+0000","last_became_active":"2024-04-25T00:28:24.170412+0000","last_became_peered":"2024-04-25T00:28:24.170412+0000","last_unstale":"2024-04-25T00:29:14.664707+0000","last_undegraded":"2024-04-25T00:29:14.664707+0000","last_fullsized":"2024-04-25T00:29:14.664707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:20.456317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431618+0000","last_change":"2024-04-25T00:28:24.182509+0000","last_active":"2024-04-25T00:29:14.431618+0000","last_peered":"2024-04-25T00:29:14.431618+0000","last_clean":"2024-04-25T00:29:14.431618+0000","last_became_active":"2024-04-25T00:28:24.182141+0000","last_became_peered":"2024-04-25T00:28:24.182141+0000","last_unstale":"2024-04-25T00:29:14.431618+0000","last_undegraded":"2024-04-25T00:29:14.431618+0000","last_fullsized":"2024-04-25T00:29:14.431618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:33.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664705+0000","last_change":"2024-04-25T00:28:24.189265+0000","last_active":"2024-04-25T00:29:14.664705+0000","last_peered":"2024-04-25T00:29:14.664705+0000","last_clean":"2024-04-25T00:29:14.664705+0000","last_became_active":"2024-04-25T00:28:24.188829+0000","last_became_peered":"2024-04-25T00:28:24.188829+0000","last_unstale":"2024-04-25T00:29:14.664705+0000","last_undegraded":"2024-04-25T00:29:14.664705+0000","last_fullsized":"2024-04-25T00:29:14.664705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:33.930373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.437356+0000","last_change":"2024-04-25T00:29:14.436642+0000","last_active":"2024-04-25T00:29:14.437356+0000","last_peered":"2024-04-25T00:29:14.437356+0000","last_clean":"2024-04-25T00:29:14.437356+0000","last_became_active":"2024-04-25T00:29:14.436291+0000","last_became_peered":"2024-04-25T00:29:14.436291+0000","last_unstale":"2024-04-25T00:29:14.437356+0000","last_undegraded":"2024-04-25T00:29:14.437356+0000","last_fullsized":"2024-04-25T00:29:14.437356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:03.844556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433007+0000","last_change":"2024-04-25T00:28:24.183769+0000","last_active":"2024-04-25T00:29:14.433007+0000","last_peered":"2024-04-25T00:29:14.433007+0000","last_clean":"2024-04-25T00:29:14.433007+0000","last_became_active":"2024-04-25T00:28:24.183432+0000","last_became_peered":"2024-04-25T00:28:24.183432+0000","last_unstale":"2024-04-25T00:29:14.433007+0000","last_undegraded":"2024-04-25T00:29:14.433007+0000","last_fullsized":"2024-04-25T00:29:14.433007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:27.656794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435041+0000","last_change":"2024-04-25T00:28:24.184364+0000","last_active":"2024-04-25T00:29:14.435041+0000","last_peered":"2024-04-25T00:29:14.435041+0000","last_clean":"2024-04-25T00:29:14.435041+0000","last_became_active":"2024-04-25T00:28:24.184038+0000","last_became_peered":"2024-04-25T00:28:24.184038+0000","last_unstale":"2024-04-25T00:29:14.435041+0000","last_undegraded":"2024-04-25T00:29:14.435041+0000","last_fullsized":"2024-04-25T00:29:14.435041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:50.495466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434953+0000","last_change":"2024-04-25T00:28:24.180236+0000","last_active":"2024-04-25T00:29:14.434953+0000","last_peered":"2024-04-25T00:29:14.434953+0000","last_clean":"2024-04-25T00:29:14.434953+0000","last_became_active":"2024-04-25T00:28:24.179929+0000","last_became_peered":"2024-04-25T00:28:24.179929+0000","last_unstale":"2024-04-25T00:29:14.434953+0000","last_undegraded":"2024-04-25T00:29:14.434953+0000","last_fullsized":"2024-04-25T00:29:14.434953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:29.858564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664086+0000","last_change":"2024-04-25T00:28:24.185605+0000","last_active":"2024-04-25T00:29:14.664086+0000","last_peered":"2024-04-25T00:29:14.664086+0000","last_clean":"2024-04-25T00:29:14.664086+0000","last_became_active":"2024-04-25T00:28:24.185270+0000","last_became_peered":"2024-04-25T00:28:24.185270+0000","last_unstale":"2024-04-25T00:29:14.664086+0000","last_undegraded":"2024-04-25T00:29:14.664086+0000","last_fullsized":"2024-04-25T00:29:14.664086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:58.798429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.437179+0000","last_change":"2024-04-25T00:28:24.187625+0000","last_active":"2024-04-25T00:29:14.437179+0000","last_peered":"2024-04-25T00:29:14.437179+0000","last_clean":"2024-04-25T00:29:14.437179+0000","last_became_active":"2024-04-25T00:28:24.186961+0000","last_became_peered":"2024-04-25T00:28:24.186961+0000","last_unstale":"2024-04-25T00:29:14.437179+0000","last_undegraded":"2024-04-25T00:29:14.437179+0000","last_fullsized":"2024-04-25T00:29:14.437179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:26.099099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429841+0000","last_change":"2024-04-25T00:28:24.177342+0000","last_active":"2024-04-25T00:29:14.429841+0000","last_peered":"2024-04-25T00:29:14.429841+0000","last_clean":"2024-04-25T00:29:14.429841+0000","last_became_active":"2024-04-25T00:28:24.176763+0000","last_became_peered":"2024-04-25T00:28:24.176763+0000","last_unstale":"2024-04-25T00:29:14.429841+0000","last_undegraded":"2024-04-25T00:29:14.429841+0000","last_fullsized":"2024-04-25T00:29:14.429841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:47.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432195+0000","last_change":"2024-04-25T00:28:24.186069+0000","last_active":"2024-04-25T00:29:14.432195+0000","last_peered":"2024-04-25T00:29:14.432195+0000","last_clean":"2024-04-25T00:29:14.432195+0000","last_became_active":"2024-04-25T00:28:24.185713+0000","last_became_peered":"2024-04-25T00:28:24.185713+0000","last_unstale":"2024-04-25T00:29:14.432195+0000","last_undegraded":"2024-04-25T00:29:14.432195+0000","last_fullsized":"2024-04-25T00:29:14.432195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:08.704071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":106,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:41.284730+0000","last_change":"2024-04-25T00:28:23.402187+0000","last_active":"2024-04-25T00:29:41.284730+0000","last_peered":"2024-04-25T00:29:41.284730+0000","last_clean":"2024-04-25T00:29:41.284730+0000","last_became_active":"2024-04-25T00:28:23.401868+0000","last_became_peered":"2024-04-25T00:28:23.401868+0000","last_unstale":"2024-04-25T00:29:41.284730+0000","last_undegraded":"2024-04-25T00:29:41.284730+0000","last_fullsized":"2024-04-25T00:29:41.284730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:44.022628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439328+0000","last_change":"2024-04-25T00:28:24.180731+0000","last_active":"2024-04-25T00:29:14.439328+0000","last_peered":"2024-04-25T00:29:14.439328+0000","last_clean":"2024-04-25T00:29:14.439328+0000","last_became_active":"2024-04-25T00:28:24.180274+0000","last_became_peered":"2024-04-25T00:28:24.180274+0000","last_unstale":"2024-04-25T00:29:14.439328+0000","last_undegraded":"2024-04-25T00:29:14.439328+0000","last_fullsized":"2024-04-25T00:29:14.439328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:46.102326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439378+0000","last_change":"2024-04-25T00:28:23.405683+0000","last_active":"2024-04-25T00:29:14.439378+0000","last_peered":"2024-04-25T00:29:14.439378+0000","last_clean":"2024-04-25T00:29:14.439378+0000","last_became_active":"2024-04-25T00:28:23.405305+0000","last_became_peered":"2024-04-25T00:28:23.405305+0000","last_unstale":"2024-04-25T00:29:14.439378+0000","last_undegraded":"2024-04-25T00:29:14.439378+0000","last_fullsized":"2024-04-25T00:29:14.439378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:39.782047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435803+0000","last_change":"2024-04-25T00:28:24.187565+0000","last_active":"2024-04-25T00:29:14.435803+0000","last_peered":"2024-04-25T00:29:14.435803+0000","last_clean":"2024-04-25T00:29:14.435803+0000","last_became_active":"2024-04-25T00:28:24.186397+0000","last_became_peered":"2024-04-25T00:28:24.186397+0000","last_unstale":"2024-04-25T00:29:14.435803+0000","last_undegraded":"2024-04-25T00:29:14.435803+0000","last_fullsized":"2024-04-25T00:29:14.435803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:36.750031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433092+0000","last_change":"2024-04-25T00:28:23.398726+0000","last_active":"2024-04-25T00:29:14.433092+0000","last_peered":"2024-04-25T00:29:14.433092+0000","last_clean":"2024-04-25T00:29:14.433092+0000","last_became_active":"2024-04-25T00:28:23.398370+0000","last_became_peered":"2024-04-25T00:28:23.398370+0000","last_unstale":"2024-04-25T00:29:14.433092+0000","last_undegraded":"2024-04-25T00:29:14.433092+0000","last_fullsized":"2024-04-25T00:29:14.433092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:19.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664297+0000","last_change":"2024-04-25T00:28:24.181424+0000","last_active":"2024-04-25T00:29:14.664297+0000","last_peered":"2024-04-25T00:29:14.664297+0000","last_clean":"2024-04-25T00:29:14.664297+0000","last_became_active":"2024-04-25T00:28:24.179077+0000","last_became_peered":"2024-04-25T00:28:24.179077+0000","last_unstale":"2024-04-25T00:29:14.664297+0000","last_undegraded":"2024-04-25T00:29:14.664297+0000","last_fullsized":"2024-04-25T00:29:14.664297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.926451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432153+0000","last_change":"2024-04-25T00:28:23.402419+0000","last_active":"2024-04-25T00:29:14.432153+0000","last_peered":"2024-04-25T00:29:14.432153+0000","last_clean":"2024-04-25T00:29:14.432153+0000","last_became_active":"2024-04-25T00:28:23.402070+0000","last_became_peered":"2024-04-25T00:28:23.402070+0000","last_unstale":"2024-04-25T00:29:14.432153+0000","last_undegraded":"2024-04-25T00:29:14.432153+0000","last_fullsized":"2024-04-25T00:29:14.432153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:08.799577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434962+0000","last_change":"2024-04-25T00:28:24.178856+0000","last_active":"2024-04-25T00:29:14.434962+0000","last_peered":"2024-04-25T00:29:14.434962+0000","last_clean":"2024-04-25T00:29:14.434962+0000","last_became_active":"2024-04-25T00:28:24.178469+0000","last_became_peered":"2024-04-25T00:28:24.178469+0000","last_unstale":"2024-04-25T00:29:14.434962+0000","last_undegraded":"2024-04-25T00:29:14.434962+0000","last_fullsized":"2024-04-25T00:29:14.434962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:28.211603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:40.266653+0000","last_change":"2024-04-25T00:28:23.400250+0000","last_active":"2024-04-25T00:29:40.266653+0000","last_peered":"2024-04-25T00:29:40.266653+0000","last_clean":"2024-04-25T00:29:40.266653+0000","last_became_active":"2024-04-25T00:28:23.399874+0000","last_became_peered":"2024-04-25T00:28:23.399874+0000","last_unstale":"2024-04-25T00:29:40.266653+0000","last_undegraded":"2024-04-25T00:29:40.266653+0000","last_fullsized":"2024-04-25T00:29:40.266653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:08.332441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432947+0000","last_change":"2024-04-25T00:28:24.170176+0000","last_active":"2024-04-25T00:29:14.432947+0000","last_peered":"2024-04-25T00:29:14.432947+0000","last_clean":"2024-04-25T00:29:14.432947+0000","last_became_active":"2024-04-25T00:28:24.169493+0000","last_became_peered":"2024-04-25T00:28:24.169493+0000","last_unstale":"2024-04-25T00:29:14.432947+0000","last_undegraded":"2024-04-25T00:29:14.432947+0000","last_fullsized":"2024-04-25T00:29:14.432947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:43.855777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.670996+0000","last_change":"2024-04-25T00:29:14.670996+0000","last_active":"2024-04-25T00:29:14.670996+0000","last_peered":"2024-04-25T00:29:14.670996+0000","last_clean":"2024-04-25T00:29:14.670996+0000","last_became_active":"2024-04-25T00:29:14.665864+0000","last_became_peered":"2024-04-25T00:29:14.665864+0000","last_unstale":"2024-04-25T00:29:14.670996+0000","last_undegraded":"2024-04-25T00:29:14.670996+0000","last_fullsized":"2024-04-25T00:29:14.670996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:18.011605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.629508+0000","last_change":"2024-04-25T00:29:14.629508+0000","last_active":"2024-04-25T00:29:14.629508+0000","last_peered":"2024-04-25T00:29:14.629508+0000","last_clean":"2024-04-25T00:29:14.629508+0000","last_became_active":"2024-04-25T00:29:14.629037+0000","last_became_peered":"2024-04-25T00:29:14.629037+0000","last_unstale":"2024-04-25T00:29:14.629508+0000","last_undegraded":"2024-04-25T00:29:14.629508+0000","last_fullsized":"2024-04-25T00:29:14.629508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:47.354462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433092+0000","last_change":"2024-04-25T00:28:23.397855+0000","last_active":"2024-04-25T00:29:14.433092+0000","last_peered":"2024-04-25T00:29:14.433092+0000","last_clean":"2024-04-25T00:29:14.433092+0000","last_became_active":"2024-04-25T00:28:23.397465+0000","last_became_peered":"2024-04-25T00:28:23.397465+0000","last_unstale":"2024-04-25T00:29:14.433092+0000","last_undegraded":"2024-04-25T00:29:14.433092+0000","last_fullsized":"2024-04-25T00:29:14.433092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.357397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439202+0000","last_change":"2024-04-25T00:28:24.172890+0000","last_active":"2024-04-25T00:29:14.439202+0000","last_peered":"2024-04-25T00:29:14.439202+0000","last_clean":"2024-04-25T00:29:14.439202+0000","last_became_active":"2024-04-25T00:28:24.172250+0000","last_became_peered":"2024-04-25T00:28:24.172250+0000","last_unstale":"2024-04-25T00:29:14.439202+0000","last_undegraded":"2024-04-25T00:29:14.439202+0000","last_fullsized":"2024-04-25T00:29:14.439202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:34.372811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434906+0000","last_change":"2024-04-25T00:28:23.401140+0000","last_active":"2024-04-25T00:29:14.434906+0000","last_peered":"2024-04-25T00:29:14.434906+0000","last_clean":"2024-04-25T00:29:14.434906+0000","last_became_active":"2024-04-25T00:28:23.400804+0000","last_became_peered":"2024-04-25T00:28:23.400804+0000","last_unstale":"2024-04-25T00:29:14.434906+0000","last_undegraded":"2024-04-25T00:29:14.434906+0000","last_fullsized":"2024-04-25T00:29:14.434906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:00.044141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429781+0000","last_change":"2024-04-25T00:28:24.175596+0000","last_active":"2024-04-25T00:29:14.429781+0000","last_peered":"2024-04-25T00:29:14.429781+0000","last_clean":"2024-04-25T00:29:14.429781+0000","last_became_active":"2024-04-25T00:28:24.175085+0000","last_became_peered":"2024-04-25T00:28:24.175085+0000","last_unstale":"2024-04-25T00:29:14.429781+0000","last_undegraded":"2024-04-25T00:29:14.429781+0000","last_fullsized":"2024-04-25T00:29:14.429781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:05.801995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:42.268990+0000","last_change":"2024-04-25T00:28:23.399427+0000","last_active":"2024-04-25T00:29:42.268990+0000","last_peered":"2024-04-25T00:29:42.268990+0000","last_clean":"2024-04-25T00:29:42.268990+0000","last_became_active":"2024-04-25T00:28:23.399067+0000","last_became_peered":"2024-04-25T00:28:23.399067+0000","last_unstale":"2024-04-25T00:29:42.268990+0000","last_undegraded":"2024-04-25T00:29:42.268990+0000","last_fullsized":"2024-04-25T00:29:42.268990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:06.853292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664998+0000","last_change":"2024-04-25T00:28:24.177367+0000","last_active":"2024-04-25T00:29:14.664998+0000","last_peered":"2024-04-25T00:29:14.664998+0000","last_clean":"2024-04-25T00:29:14.664998+0000","last_became_active":"2024-04-25T00:28:24.176890+0000","last_became_peered":"2024-04-25T00:28:24.176890+0000","last_unstale":"2024-04-25T00:29:14.664998+0000","last_undegraded":"2024-04-25T00:29:14.664998+0000","last_fullsized":"2024-04-25T00:29:14.664998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:27.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664639+0000","last_change":"2024-04-25T00:28:23.409705+0000","last_active":"2024-04-25T00:29:14.664639+0000","last_peered":"2024-04-25T00:29:14.664639+0000","last_clean":"2024-04-25T00:29:14.664639+0000","last_became_active":"2024-04-25T00:28:23.408596+0000","last_became_peered":"2024-04-25T00:28:23.408596+0000","last_unstale":"2024-04-25T00:29:14.664639+0000","last_undegraded":"2024-04-25T00:29:14.664639+0000","last_fullsized":"2024-04-25T00:29:14.664639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:25.684497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435704+0000","last_change":"2024-04-25T00:28:24.179361+0000","last_active":"2024-04-25T00:29:14.435704+0000","last_peered":"2024-04-25T00:29:14.435704+0000","last_clean":"2024-04-25T00:29:14.435704+0000","last_became_active":"2024-04-25T00:28:24.179019+0000","last_became_peered":"2024-04-25T00:28:24.179019+0000","last_unstale":"2024-04-25T00:29:14.435704+0000","last_undegraded":"2024-04-25T00:29:14.435704+0000","last_fullsized":"2024-04-25T00:29:14.435704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:08.543283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435090+0000","last_change":"2024-04-25T00:28:23.404051+0000","last_active":"2024-04-25T00:29:14.435090+0000","last_peered":"2024-04-25T00:29:14.435090+0000","last_clean":"2024-04-25T00:29:14.435090+0000","last_became_active":"2024-04-25T00:28:23.403728+0000","last_became_peered":"2024-04-25T00:28:23.403728+0000","last_unstale":"2024-04-25T00:29:14.435090+0000","last_undegraded":"2024-04-25T00:29:14.435090+0000","last_fullsized":"2024-04-25T00:29:14.435090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:27.403464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664511+0000","last_change":"2024-04-25T00:28:24.178506+0000","last_active":"2024-04-25T00:29:14.664511+0000","last_peered":"2024-04-25T00:29:14.664511+0000","last_clean":"2024-04-25T00:29:14.664511+0000","last_became_active":"2024-04-25T00:28:24.177053+0000","last_became_peered":"2024-04-25T00:28:24.177053+0000","last_unstale":"2024-04-25T00:29:14.664511+0000","last_undegraded":"2024-04-25T00:29:14.664511+0000","last_fullsized":"2024-04-25T00:29:14.664511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:04.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:42.285951+0000","last_change":"2024-04-25T00:28:23.400563+0000","last_active":"2024-04-25T00:29:42.285951+0000","last_peered":"2024-04-25T00:29:42.285951+0000","last_clean":"2024-04-25T00:29:42.285951+0000","last_became_active":"2024-04-25T00:28:23.399982+0000","last_became_peered":"2024-04-25T00:28:23.399982+0000","last_unstale":"2024-04-25T00:29:42.285951+0000","last_undegraded":"2024-04-25T00:29:42.285951+0000","last_fullsized":"2024-04-25T00:29:42.285951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:01.652027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":136,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433951+0000","last_change":"2024-04-25T00:28:24.179687+0000","last_active":"2024-04-25T00:29:14.433951+0000","last_peered":"2024-04-25T00:29:14.433951+0000","last_clean":"2024-04-25T00:29:14.433951+0000","last_became_active":"2024-04-25T00:28:24.179345+0000","last_became_peered":"2024-04-25T00:28:24.179345+0000","last_unstale":"2024-04-25T00:29:14.433951+0000","last_undegraded":"2024-04-25T00:29:14.433951+0000","last_fullsized":"2024-04-25T00:29:14.433951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:49.807417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432873+0000","last_change":"2024-04-25T00:28:23.401353+0000","last_active":"2024-04-25T00:29:14.432873+0000","last_peered":"2024-04-25T00:29:14.432873+0000","last_clean":"2024-04-25T00:29:14.432873+0000","last_became_active":"2024-04-25T00:28:23.400985+0000","last_became_peered":"2024-04-25T00:28:23.400985+0000","last_unstale":"2024-04-25T00:29:14.432873+0000","last_undegraded":"2024-04-25T00:29:14.432873+0000","last_fullsized":"2024-04-25T00:29:14.432873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.277514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432818+0000","last_change":"2024-04-25T00:28:24.178796+0000","last_active":"2024-04-25T00:29:14.432818+0000","last_peered":"2024-04-25T00:29:14.432818+0000","last_clean":"2024-04-25T00:29:14.432818+0000","last_became_active":"2024-04-25T00:28:24.178457+0000","last_became_peered":"2024-04-25T00:28:24.178457+0000","last_unstale":"2024-04-25T00:29:14.432818+0000","last_undegraded":"2024-04-25T00:29:14.432818+0000","last_fullsized":"2024-04-25T00:29:14.432818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:48.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434395+0000","last_change":"2024-04-25T00:28:23.406046+0000","last_active":"2024-04-25T00:29:14.434395+0000","last_peered":"2024-04-25T00:29:14.434395+0000","last_clean":"2024-04-25T00:29:14.434395+0000","last_became_active":"2024-04-25T00:28:23.405686+0000","last_became_peered":"2024-04-25T00:28:23.405686+0000","last_unstale":"2024-04-25T00:29:14.434395+0000","last_undegraded":"2024-04-25T00:29:14.434395+0000","last_fullsized":"2024-04-25T00:29:14.434395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:30.551884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664511+0000","last_change":"2024-04-25T00:28:24.188251+0000","last_active":"2024-04-25T00:29:14.664511+0000","last_peered":"2024-04-25T00:29:14.664511+0000","last_clean":"2024-04-25T00:29:14.664511+0000","last_became_active":"2024-04-25T00:28:24.187876+0000","last_became_peered":"2024-04-25T00:28:24.187876+0000","last_unstale":"2024-04-25T00:29:14.664511+0000","last_undegraded":"2024-04-25T00:29:14.664511+0000","last_fullsized":"2024-04-25T00:29:14.664511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:18.269074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:41.267345+0000","last_change":"2024-04-25T00:28:23.407240+0000","last_active":"2024-04-25T00:29:41.267345+0000","last_peered":"2024-04-25T00:29:41.267345+0000","last_clean":"2024-04-25T00:29:41.267345+0000","last_became_active":"2024-04-25T00:28:23.406794+0000","last_became_peered":"2024-04-25T00:28:23.406794+0000","last_unstale":"2024-04-25T00:29:41.267345+0000","last_undegraded":"2024-04-25T00:29:41.267345+0000","last_fullsized":"2024-04-25T00:29:41.267345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:35.861733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.663856+0000","last_change":"2024-04-25T00:28:24.183784+0000","last_active":"2024-04-25T00:29:14.663856+0000","last_peered":"2024-04-25T00:29:14.663856+0000","last_clean":"2024-04-25T00:29:14.663856+0000","last_became_active":"2024-04-25T00:28:24.183417+0000","last_became_peered":"2024-04-25T00:28:24.183417+0000","last_unstale":"2024-04-25T00:29:14.663856+0000","last_undegraded":"2024-04-25T00:29:14.663856+0000","last_fullsized":"2024-04-25T00:29:14.663856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:43.182050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.642117+0000","last_change":"2024-04-25T00:29:14.642117+0000","last_active":"2024-04-25T00:29:14.642117+0000","last_peered":"2024-04-25T00:29:14.642117+0000","last_clean":"2024-04-25T00:29:14.642117+0000","last_became_active":"2024-04-25T00:29:14.629616+0000","last_became_peered":"2024-04-25T00:29:14.629616+0000","last_unstale":"2024-04-25T00:29:14.642117+0000","last_undegraded":"2024-04-25T00:29:14.642117+0000","last_fullsized":"2024-04-25T00:29:14.642117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:50.342238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434395+0000","last_change":"2024-04-25T00:28:24.177898+0000","last_active":"2024-04-25T00:29:14.434395+0000","last_peered":"2024-04-25T00:29:14.434395+0000","last_clean":"2024-04-25T00:29:14.434395+0000","last_became_active":"2024-04-25T00:28:24.177578+0000","last_became_peered":"2024-04-25T00:28:24.177578+0000","last_unstale":"2024-04-25T00:29:14.434395+0000","last_undegraded":"2024-04-25T00:29:14.434395+0000","last_fullsized":"2024-04-25T00:29:14.434395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:36.294161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432401+0000","last_change":"2024-04-25T00:28:23.399570+0000","last_active":"2024-04-25T00:29:14.432401+0000","last_peered":"2024-04-25T00:29:14.432401+0000","last_clean":"2024-04-25T00:29:14.432401+0000","last_became_active":"2024-04-25T00:28:23.399230+0000","last_became_peered":"2024-04-25T00:28:23.399230+0000","last_unstale":"2024-04-25T00:29:14.432401+0000","last_undegraded":"2024-04-25T00:29:14.432401+0000","last_fullsized":"2024-04-25T00:29:14.432401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:46.850590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.436728+0000","last_change":"2024-04-25T00:28:24.186113+0000","last_active":"2024-04-25T00:29:14.436728+0000","last_peered":"2024-04-25T00:29:14.436728+0000","last_clean":"2024-04-25T00:29:14.436728+0000","last_became_active":"2024-04-25T00:28:24.185743+0000","last_became_peered":"2024-04-25T00:28:24.185743+0000","last_unstale":"2024-04-25T00:29:14.436728+0000","last_undegraded":"2024-04-25T00:29:14.436728+0000","last_fullsized":"2024-04-25T00:29:14.436728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:10.973285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429923+0000","last_change":"2024-04-25T00:28:23.396054+0000","last_active":"2024-04-25T00:29:14.429923+0000","last_peered":"2024-04-25T00:29:14.429923+0000","last_clean":"2024-04-25T00:29:14.429923+0000","last_became_active":"2024-04-25T00:28:23.395707+0000","last_became_peered":"2024-04-25T00:28:23.395707+0000","last_unstale":"2024-04-25T00:29:14.429923+0000","last_undegraded":"2024-04-25T00:29:14.429923+0000","last_fullsized":"2024-04-25T00:29:14.429923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:20.169476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431543+0000","last_change":"2024-04-25T00:28:24.185655+0000","last_active":"2024-04-25T00:29:14.431543+0000","last_peered":"2024-04-25T00:29:14.431543+0000","last_clean":"2024-04-25T00:29:14.431543+0000","last_became_active":"2024-04-25T00:28:24.185123+0000","last_became_peered":"2024-04-25T00:28:24.185123+0000","last_unstale":"2024-04-25T00:29:14.431543+0000","last_undegraded":"2024-04-25T00:29:14.431543+0000","last_fullsized":"2024-04-25T00:29:14.431543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.845762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:38.281756+0000","last_change":"2024-04-25T00:28:23.410964+0000","last_active":"2024-04-25T00:29:38.281756+0000","last_peered":"2024-04-25T00:29:38.281756+0000","last_clean":"2024-04-25T00:29:38.281756+0000","last_became_active":"2024-04-25T00:28:23.410602+0000","last_became_peered":"2024-04-25T00:28:23.410602+0000","last_unstale":"2024-04-25T00:29:38.281756+0000","last_undegraded":"2024-04-25T00:29:38.281756+0000","last_fullsized":"2024-04-25T00:29:38.281756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:24.495790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431458+0000","last_change":"2024-04-25T00:28:24.184710+0000","last_active":"2024-04-25T00:29:14.431458+0000","last_peered":"2024-04-25T00:29:14.431458+0000","last_clean":"2024-04-25T00:29:14.431458+0000","last_became_active":"2024-04-25T00:28:24.184354+0000","last_became_peered":"2024-04-25T00:28:24.184354+0000","last_unstale":"2024-04-25T00:29:14.431458+0000","last_undegraded":"2024-04-25T00:29:14.431458+0000","last_fullsized":"2024-04-25T00:29:14.431458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:47.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.438916+0000","last_change":"2024-04-25T00:29:14.438916+0000","last_active":"2024-04-25T00:29:14.438916+0000","last_peered":"2024-04-25T00:29:14.438916+0000","last_clean":"2024-04-25T00:29:14.438916+0000","last_became_active":"2024-04-25T00:29:14.438376+0000","last_became_peered":"2024-04-25T00:29:14.438376+0000","last_unstale":"2024-04-25T00:29:14.438916+0000","last_undegraded":"2024-04-25T00:29:14.438916+0000","last_fullsized":"2024-04-25T00:29:14.438916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:58.938866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:37.261258+0000","last_change":"2024-04-25T00:28:23.401437+0000","last_active":"2024-04-25T00:29:37.261258+0000","last_peered":"2024-04-25T00:29:37.261258+0000","last_clean":"2024-04-25T00:29:37.261258+0000","last_became_active":"2024-04-25T00:28:23.401091+0000","last_became_peered":"2024-04-25T00:28:23.401091+0000","last_unstale":"2024-04-25T00:29:37.261258+0000","last_undegraded":"2024-04-25T00:29:37.261258+0000","last_fullsized":"2024-04-25T00:29:37.261258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:58.748461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.440348+0000","last_change":"2024-04-25T00:28:24.166224+0000","last_active":"2024-04-25T00:29:14.440348+0000","last_peered":"2024-04-25T00:29:14.440348+0000","last_clean":"2024-04-25T00:29:14.440348+0000","last_became_active":"2024-04-25T00:28:24.165626+0000","last_became_peered":"2024-04-25T00:28:24.165626+0000","last_unstale":"2024-04-25T00:29:14.440348+0000","last_undegraded":"2024-04-25T00:29:14.440348+0000","last_fullsized":"2024-04-25T00:29:14.440348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:04.956168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.628121+0000","last_change":"2024-04-25T00:28:23.408551+0000","last_active":"2024-04-25T00:29:14.628121+0000","last_peered":"2024-04-25T00:29:14.628121+0000","last_clean":"2024-04-25T00:29:14.628121+0000","last_became_active":"2024-04-25T00:28:23.408137+0000","last_became_peered":"2024-04-25T00:28:23.408137+0000","last_unstale":"2024-04-25T00:29:14.628121+0000","last_undegraded":"2024-04-25T00:29:14.628121+0000","last_fullsized":"2024-04-25T00:29:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:53.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.663940+0000","last_change":"2024-04-25T00:27:10.587192+0000","last_active":"2024-04-25T00:29:14.663940+0000","last_peered":"2024-04-25T00:29:14.663940+0000","last_clean":"2024-04-25T00:29:14.663940+0000","last_became_active":"2024-04-25T00:27:10.586481+0000","last_became_peered":"2024-04-25T00:27:10.586481+0000","last_unstale":"2024-04-25T00:29:14.663940+0000","last_undegraded":"2024-04-25T00:29:14.663940+0000","last_fullsized":"2024-04-25T00:29:14.663940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_clean_scrub_stamp":"2024-04-25T00:23:42.305887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:44.337443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.437268+0000","last_change":"2024-04-25T00:29:14.437268+0000","last_active":"2024-04-25T00:29:14.437268+0000","last_peered":"2024-04-25T00:29:14.437268+0000","last_clean":"2024-04-25T00:29:14.437268+0000","last_became_active":"2024-04-25T00:29:14.436284+0000","last_became_peered":"2024-04-25T00:29:14.436284+0000","last_unstale":"2024-04-25T00:29:14.437268+0000","last_undegraded":"2024-04-25T00:29:14.437268+0000","last_fullsized":"2024-04-25T00:29:14.437268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:42.576164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664050+0000","last_change":"2024-04-25T00:28:23.404002+0000","last_active":"2024-04-25T00:29:14.664050+0000","last_peered":"2024-04-25T00:29:14.664050+0000","last_clean":"2024-04-25T00:29:14.664050+0000","last_became_active":"2024-04-25T00:28:23.403659+0000","last_became_peered":"2024-04-25T00:28:23.403659+0000","last_unstale":"2024-04-25T00:29:14.664050+0000","last_undegraded":"2024-04-25T00:29:14.664050+0000","last_fullsized":"2024-04-25T00:29:14.664050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:26.994242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432875+0000","last_change":"2024-04-25T00:28:23.405525+0000","last_active":"2024-04-25T00:29:14.432875+0000","last_peered":"2024-04-25T00:29:14.432875+0000","last_clean":"2024-04-25T00:29:14.432875+0000","last_became_active":"2024-04-25T00:28:23.405228+0000","last_became_peered":"2024-04-25T00:28:23.405228+0000","last_unstale":"2024-04-25T00:29:14.432875+0000","last_undegraded":"2024-04-25T00:29:14.432875+0000","last_fullsized":"2024-04-25T00:29:14.432875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:00.819210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433102+0000","last_change":"2024-04-25T00:28:24.182350+0000","last_active":"2024-04-25T00:29:14.433102+0000","last_peered":"2024-04-25T00:29:14.433102+0000","last_clean":"2024-04-25T00:29:14.433102+0000","last_became_active":"2024-04-25T00:28:24.182002+0000","last_became_peered":"2024-04-25T00:28:24.182002+0000","last_unstale":"2024-04-25T00:29:14.433102+0000","last_undegraded":"2024-04-25T00:29:14.433102+0000","last_fullsized":"2024-04-25T00:29:14.433102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:55.891046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429957+0000","last_change":"2024-04-25T00:28:23.405357+0000","last_active":"2024-04-25T00:29:14.429957+0000","last_peered":"2024-04-25T00:29:14.429957+0000","last_clean":"2024-04-25T00:29:14.429957+0000","last_became_active":"2024-04-25T00:28:23.405031+0000","last_became_peered":"2024-04-25T00:28:23.405031+0000","last_unstale":"2024-04-25T00:29:14.429957+0000","last_undegraded":"2024-04-25T00:29:14.429957+0000","last_fullsized":"2024-04-25T00:29:14.429957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:10.683462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439561+0000","last_change":"2024-04-25T00:28:24.177859+0000","last_active":"2024-04-25T00:29:14.439561+0000","last_peered":"2024-04-25T00:29:14.439561+0000","last_clean":"2024-04-25T00:29:14.439561+0000","last_became_active":"2024-04-25T00:28:24.177533+0000","last_became_peered":"2024-04-25T00:28:24.177533+0000","last_unstale":"2024-04-25T00:29:14.439561+0000","last_undegraded":"2024-04-25T00:29:14.439561+0000","last_fullsized":"2024-04-25T00:29:14.439561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:46.964934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.628107+0000","last_change":"2024-04-25T00:28:23.409487+0000","last_active":"2024-04-25T00:29:14.628107+0000","last_peered":"2024-04-25T00:29:14.628107+0000","last_clean":"2024-04-25T00:29:14.628107+0000","last_became_active":"2024-04-25T00:28:23.409146+0000","last_became_peered":"2024-04-25T00:28:23.409146+0000","last_unstale":"2024-04-25T00:29:14.628107+0000","last_undegraded":"2024-04-25T00:29:14.628107+0000","last_fullsized":"2024-04-25T00:29:14.628107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.591409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435503+0000","last_change":"2024-04-25T00:28:24.185083+0000","last_active":"2024-04-25T00:29:14.435503+0000","last_peered":"2024-04-25T00:29:14.435503+0000","last_clean":"2024-04-25T00:29:14.435503+0000","last_became_active":"2024-04-25T00:28:24.184752+0000","last_became_peered":"2024-04-25T00:28:24.184752+0000","last_unstale":"2024-04-25T00:29:14.435503+0000","last_undegraded":"2024-04-25T00:29:14.435503+0000","last_fullsized":"2024-04-25T00:29:14.435503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:08.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664144+0000","last_change":"2024-04-25T00:28:23.405662+0000","last_active":"2024-04-25T00:29:14.664144+0000","last_peered":"2024-04-25T00:29:14.664144+0000","last_clean":"2024-04-25T00:29:14.664144+0000","last_became_active":"2024-04-25T00:28:23.404949+0000","last_became_peered":"2024-04-25T00:28:23.404949+0000","last_unstale":"2024-04-25T00:29:14.664144+0000","last_undegraded":"2024-04-25T00:29:14.664144+0000","last_fullsized":"2024-04-25T00:29:14.664144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:01.989041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431610+0000","last_change":"2024-04-25T00:28:24.181404+0000","last_active":"2024-04-25T00:29:14.431610+0000","last_peered":"2024-04-25T00:29:14.431610+0000","last_clean":"2024-04-25T00:29:14.431610+0000","last_became_active":"2024-04-25T00:28:24.180399+0000","last_became_peered":"2024-04-25T00:28:24.180399+0000","last_unstale":"2024-04-25T00:29:14.431610+0000","last_undegraded":"2024-04-25T00:29:14.431610+0000","last_fullsized":"2024-04-25T00:29:14.431610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:41.955204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439444+0000","last_change":"2024-04-25T00:28:23.405288+0000","last_active":"2024-04-25T00:29:14.439444+0000","last_peered":"2024-04-25T00:29:14.439444+0000","last_clean":"2024-04-25T00:29:14.439444+0000","last_became_active":"2024-04-25T00:28:23.404547+0000","last_became_peered":"2024-04-25T00:28:23.404547+0000","last_unstale":"2024-04-25T00:29:14.439444+0000","last_undegraded":"2024-04-25T00:29:14.439444+0000","last_fullsized":"2024-04-25T00:29:14.439444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:45.180422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435974+0000","last_change":"2024-04-25T00:28:24.176840+0000","last_active":"2024-04-25T00:29:14.435974+0000","last_peered":"2024-04-25T00:29:14.435974+0000","last_clean":"2024-04-25T00:29:14.435974+0000","last_became_active":"2024-04-25T00:28:24.176478+0000","last_became_peered":"2024-04-25T00:28:24.176478+0000","last_unstale":"2024-04-25T00:29:14.435974+0000","last_undegraded":"2024-04-25T00:29:14.435974+0000","last_fullsized":"2024-04-25T00:29:14.435974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:50.057929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.447278+0000","last_change":"2024-04-25T00:29:14.447278+0000","last_active":"2024-04-25T00:29:14.447278+0000","last_peered":"2024-04-25T00:29:14.447278+0000","last_clean":"2024-04-25T00:29:14.447278+0000","last_became_active":"2024-04-25T00:29:14.440971+0000","last_became_peered":"2024-04-25T00:29:14.440971+0000","last_unstale":"2024-04-25T00:29:14.447278+0000","last_undegraded":"2024-04-25T00:29:14.447278+0000","last_fullsized":"2024-04-25T00:29:14.447278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.256673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.440035+0000","last_change":"2024-04-25T00:28:24.176402+0000","last_active":"2024-04-25T00:29:14.440035+0000","last_peered":"2024-04-25T00:29:14.440035+0000","last_clean":"2024-04-25T00:29:14.440035+0000","last_became_active":"2024-04-25T00:28:24.175822+0000","last_became_peered":"2024-04-25T00:28:24.175822+0000","last_unstale":"2024-04-25T00:29:14.440035+0000","last_undegraded":"2024-04-25T00:29:14.440035+0000","last_fullsized":"2024-04-25T00:29:14.440035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.952116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433259+0000","last_change":"2024-04-25T00:28:23.401158+0000","last_active":"2024-04-25T00:29:14.433259+0000","last_peered":"2024-04-25T00:29:14.433259+0000","last_clean":"2024-04-25T00:29:14.433259+0000","last_became_active":"2024-04-25T00:28:23.400819+0000","last_became_peered":"2024-04-25T00:28:23.400819+0000","last_unstale":"2024-04-25T00:29:14.433259+0000","last_undegraded":"2024-04-25T00:29:14.433259+0000","last_fullsized":"2024-04-25T00:29:14.433259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.209636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664451+0000","last_change":"2024-04-25T00:28:24.189784+0000","last_active":"2024-04-25T00:29:14.664451+0000","last_peered":"2024-04-25T00:29:14.664451+0000","last_clean":"2024-04-25T00:29:14.664451+0000","last_became_active":"2024-04-25T00:28:24.188161+0000","last_became_peered":"2024-04-25T00:28:24.188161+0000","last_unstale":"2024-04-25T00:29:14.664451+0000","last_undegraded":"2024-04-25T00:29:14.664451+0000","last_fullsized":"2024-04-25T00:29:14.664451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:06.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:40.285867+0000","last_change":"2024-04-25T00:28:23.410684+0000","last_active":"2024-04-25T00:29:40.285867+0000","last_peered":"2024-04-25T00:29:40.285867+0000","last_clean":"2024-04-25T00:29:40.285867+0000","last_became_active":"2024-04-25T00:28:23.410345+0000","last_became_peered":"2024-04-25T00:28:23.410345+0000","last_unstale":"2024-04-25T00:29:40.285867+0000","last_undegraded":"2024-04-25T00:29:40.285867+0000","last_fullsized":"2024-04-25T00:29:40.285867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664820+0000","last_change":"2024-04-25T00:28:24.189170+0000","last_active":"2024-04-25T00:29:14.664820+0000","last_peered":"2024-04-25T00:29:14.664820+0000","last_clean":"2024-04-25T00:29:14.664820+0000","last_became_active":"2024-04-25T00:28:24.188329+0000","last_became_peered":"2024-04-25T00:28:24.188329+0000","last_unstale":"2024-04-25T00:29:14.664820+0000","last_undegraded":"2024-04-25T00:29:14.664820+0000","last_fullsized":"2024-04-25T00:29:14.664820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:41.523820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434851+0000","last_change":"2024-04-25T00:28:23.405933+0000","last_active":"2024-04-25T00:29:14.434851+0000","last_peered":"2024-04-25T00:29:14.434851+0000","last_clean":"2024-04-25T00:29:14.434851+0000","last_became_active":"2024-04-25T00:28:23.405585+0000","last_became_peered":"2024-04-25T00:28:23.405585+0000","last_unstale":"2024-04-25T00:29:14.434851+0000","last_undegraded":"2024-04-25T00:29:14.434851+0000","last_fullsized":"2024-04-25T00:29:14.434851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:05.914735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664033+0000","last_change":"2024-04-25T00:28:24.178720+0000","last_active":"2024-04-25T00:29:14.664033+0000","last_peered":"2024-04-25T00:29:14.664033+0000","last_clean":"2024-04-25T00:29:14.664033+0000","last_became_active":"2024-04-25T00:28:24.178270+0000","last_became_peered":"2024-04-25T00:28:24.178270+0000","last_unstale":"2024-04-25T00:29:14.664033+0000","last_undegraded":"2024-04-25T00:29:14.664033+0000","last_fullsized":"2024-04-25T00:29:14.664033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:38.076554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433230+0000","last_change":"2024-04-25T00:28:23.403017+0000","last_active":"2024-04-25T00:29:14.433230+0000","last_peered":"2024-04-25T00:29:14.433230+0000","last_clean":"2024-04-25T00:29:14.433230+0000","last_became_active":"2024-04-25T00:28:23.402706+0000","last_became_peered":"2024-04-25T00:28:23.402706+0000","last_unstale":"2024-04-25T00:29:14.433230+0000","last_undegraded":"2024-04-25T00:29:14.433230+0000","last_fullsized":"2024-04-25T00:29:14.433230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:06.916509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664153+0000","last_change":"2024-04-25T00:28:24.181585+0000","last_active":"2024-04-25T00:29:14.664153+0000","last_peered":"2024-04-25T00:29:14.664153+0000","last_clean":"2024-04-25T00:29:14.664153+0000","last_became_active":"2024-04-25T00:28:24.180755+0000","last_became_peered":"2024-04-25T00:28:24.180755+0000","last_unstale":"2024-04-25T00:29:14.664153+0000","last_undegraded":"2024-04-25T00:29:14.664153+0000","last_fullsized":"2024-04-25T00:29:14.664153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:59.711999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.436099+0000","last_change":"2024-04-25T00:28:24.175444+0000","last_active":"2024-04-25T00:29:14.436099+0000","last_peered":"2024-04-25T00:29:14.436099+0000","last_clean":"2024-04-25T00:29:14.436099+0000","last_became_active":"2024-04-25T00:28:24.174322+0000","last_became_peered":"2024-04-25T00:28:24.174322+0000","last_unstale":"2024-04-25T00:29:14.436099+0000","last_undegraded":"2024-04-25T00:29:14.436099+0000","last_fullsized":"2024-04-25T00:29:14.436099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:15.325122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432339+0000","last_change":"2024-04-25T00:28:24.188603+0000","last_active":"2024-04-25T00:29:14.432339+0000","last_peered":"2024-04-25T00:29:14.432339+0000","last_clean":"2024-04-25T00:29:14.432339+0000","last_became_active":"2024-04-25T00:28:24.188300+0000","last_became_peered":"2024-04-25T00:28:24.188300+0000","last_unstale":"2024-04-25T00:29:14.432339+0000","last_undegraded":"2024-04-25T00:29:14.432339+0000","last_fullsized":"2024-04-25T00:29:14.432339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:45.466914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431821+0000","last_change":"2024-04-25T00:28:24.183907+0000","last_active":"2024-04-25T00:29:14.431821+0000","last_peered":"2024-04-25T00:29:14.431821+0000","last_clean":"2024-04-25T00:29:14.431821+0000","last_became_active":"2024-04-25T00:28:24.183545+0000","last_became_peered":"2024-04-25T00:28:24.183545+0000","last_unstale":"2024-04-25T00:29:14.431821+0000","last_undegraded":"2024-04-25T00:29:14.431821+0000","last_fullsized":"2024-04-25T00:29:14.431821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:02.883338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435288+0000","last_change":"2024-04-25T00:28:24.178516+0000","last_active":"2024-04-25T00:29:14.435288+0000","last_peered":"2024-04-25T00:29:14.435288+0000","last_clean":"2024-04-25T00:29:14.435288+0000","last_became_active":"2024-04-25T00:28:24.177995+0000","last_became_peered":"2024-04-25T00:28:24.177995+0000","last_unstale":"2024-04-25T00:29:14.435288+0000","last_undegraded":"2024-04-25T00:29:14.435288+0000","last_fullsized":"2024-04-25T00:29:14.435288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:03.396880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433173+0000","last_change":"2024-04-25T00:28:24.175401+0000","last_active":"2024-04-25T00:29:14.433173+0000","last_peered":"2024-04-25T00:29:14.433173+0000","last_clean":"2024-04-25T00:29:14.433173+0000","last_became_active":"2024-04-25T00:28:24.174648+0000","last_became_peered":"2024-04-25T00:28:24.174648+0000","last_unstale":"2024-04-25T00:29:14.433173+0000","last_undegraded":"2024-04-25T00:29:14.433173+0000","last_fullsized":"2024-04-25T00:29:14.433173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:00.630810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.436669+0000","last_change":"2024-04-25T00:28:24.173542+0000","last_active":"2024-04-25T00:29:14.436669+0000","last_peered":"2024-04-25T00:29:14.436669+0000","last_clean":"2024-04-25T00:29:14.436669+0000","last_became_active":"2024-04-25T00:28:24.173240+0000","last_became_peered":"2024-04-25T00:28:24.173240+0000","last_unstale":"2024-04-25T00:29:14.436669+0000","last_undegraded":"2024-04-25T00:29:14.436669+0000","last_fullsized":"2024-04-25T00:29:14.436669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:00.189612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432561+0000","last_change":"2024-04-25T00:28:24.185104+0000","last_active":"2024-04-25T00:29:14.432561+0000","last_peered":"2024-04-25T00:29:14.432561+0000","last_clean":"2024-04-25T00:29:14.432561+0000","last_became_active":"2024-04-25T00:28:24.184566+0000","last_became_peered":"2024-04-25T00:28:24.184566+0000","last_unstale":"2024-04-25T00:29:14.432561+0000","last_undegraded":"2024-04-25T00:29:14.432561+0000","last_fullsized":"2024-04-25T00:29:14.432561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:48.469722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439474+0000","last_change":"2024-04-25T00:28:24.181286+0000","last_active":"2024-04-25T00:29:14.439474+0000","last_peered":"2024-04-25T00:29:14.439474+0000","last_clean":"2024-04-25T00:29:14.439474+0000","last_became_active":"2024-04-25T00:28:24.180406+0000","last_became_peered":"2024-04-25T00:28:24.180406+0000","last_unstale":"2024-04-25T00:29:14.439474+0000","last_undegraded":"2024-04-25T00:29:14.439474+0000","last_fullsized":"2024-04-25T00:29:14.439474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.644228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.441147+0000","last_change":"2024-04-25T00:29:14.440689+0000","last_active":"2024-04-25T00:29:14.441147+0000","last_peered":"2024-04-25T00:29:14.441147+0000","last_clean":"2024-04-25T00:29:14.441147+0000","last_became_active":"2024-04-25T00:29:14.440255+0000","last_became_peered":"2024-04-25T00:29:14.440255+0000","last_unstale":"2024-04-25T00:29:14.441147+0000","last_undegraded":"2024-04-25T00:29:14.441147+0000","last_fullsized":"2024-04-25T00:29:14.441147+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:37.142787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439321+0000","last_change":"2024-04-25T00:28:24.180521+0000","last_active":"2024-04-25T00:29:14.439321+0000","last_peered":"2024-04-25T00:29:14.439321+0000","last_clean":"2024-04-25T00:29:14.439321+0000","last_became_active":"2024-04-25T00:28:24.179867+0000","last_became_peered":"2024-04-25T00:28:24.179867+0000","last_unstale":"2024-04-25T00:29:14.439321+0000","last_undegraded":"2024-04-25T00:29:14.439321+0000","last_fullsized":"2024-04-25T00:29:14.439321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:31.660585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664698+0000","last_change":"2024-04-25T00:28:24.185050+0000","last_active":"2024-04-25T00:29:14.664698+0000","last_peered":"2024-04-25T00:29:14.664698+0000","last_clean":"2024-04-25T00:29:14.664698+0000","last_became_active":"2024-04-25T00:28:24.184663+0000","last_became_peered":"2024-04-25T00:28:24.184663+0000","last_unstale":"2024-04-25T00:29:14.664698+0000","last_undegraded":"2024-04-25T00:29:14.664698+0000","last_fullsized":"2024-04-25T00:29:14.664698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:32.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432947+0000","last_change":"2024-04-25T00:28:24.177948+0000","last_active":"2024-04-25T00:29:14.432947+0000","last_peered":"2024-04-25T00:29:14.432947+0000","last_clean":"2024-04-25T00:29:14.432947+0000","last_became_active":"2024-04-25T00:28:24.177627+0000","last_became_peered":"2024-04-25T00:28:24.177627+0000","last_unstale":"2024-04-25T00:29:14.432947+0000","last_undegraded":"2024-04-25T00:29:14.432947+0000","last_fullsized":"2024-04-25T00:29:14.432947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:29.128868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.436364+0000","last_change":"2024-04-25T00:28:24.176115+0000","last_active":"2024-04-25T00:29:14.436364+0000","last_peered":"2024-04-25T00:29:14.436364+0000","last_clean":"2024-04-25T00:29:14.436364+0000","last_became_active":"2024-04-25T00:28:24.175735+0000","last_became_peered":"2024-04-25T00:28:24.175735+0000","last_unstale":"2024-04-25T00:29:14.436364+0000","last_undegraded":"2024-04-25T00:29:14.436364+0000","last_fullsized":"2024-04-25T00:29:14.436364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:58.595542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280907,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66836,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":772,"num_read_kb":459,"num_write":237,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318299,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":465335,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1116387,"data_compressed":41565,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":465265,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005101,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":465643,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1058161,"data_compressed":41565,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":464982,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":525918,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":466321,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":526417,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":464807,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":464955,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1057839,"data_compressed":41565,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:29:45.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:29:45.461 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:45.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:45 smithi031 ceph-mon[29167]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:29:46.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:45 smithi169 ceph-mon[33112]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:29:46.144 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:45 smithi040 ceph-mon[32945]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:29:47.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:46 smithi031 ceph-mon[29167]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:47.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:46 smithi169 ceph-mon[33112]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:47.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:46 smithi040 ceph-mon[32945]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 -- 172.21.15.31:0/3867528087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1008f0 msgr2=0x7f612c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 --2- 172.21.15.31:0/3867528087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1008f0 0x7f612c100d70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f612800aaa0 tx=0x7f612800adb0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 -- 172.21.15.31:0/3867528087 shutdown_connections 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 --2- 172.21.15.31:0/3867528087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f612c1012b0 0x7f612c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 --2- 172.21.15.31:0/3867528087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f612c1068a0 0x7f612c106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 --2- 172.21.15.31:0/3867528087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1008f0 0x7f612c100d70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 -- 172.21.15.31:0/3867528087 >> 172.21.15.31:0/3867528087 conn(0x7f612c0777a0 msgr2=0x7f612c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 -- 172.21.15.31:0/3867528087 shutdown_connections 2024-04-25T00:29:47.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.273+0000 7f6133bae700 1 -- 172.21.15.31:0/3867528087 wait complete. 2024-04-25T00:29:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 Processor -- start 2024-04-25T00:29:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 -- start start 2024-04-25T00:29:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f612c1008f0 0x7f612c0fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f612c1012b0 0x7f612c0fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 0x7f612c0fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f612c110270 con 0x7f612c1068a0 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1100f0 con 0x7f612c1012b0 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6133bae700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1103f0 con 0x7f612c1008f0 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f613214b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 0x7f612c0fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f613214b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 0x7f612c0fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55538/0 (socket says 172.21.15.31:55538) 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f613214b700 1 -- 172.21.15.31:0/2643837816 learned_addr learned my addr 172.21.15.31:0/2643837816 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f6131149700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f612c1012b0 0x7f612c0fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.274+0000 7f613194a700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f612c1008f0 0x7f612c0fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f613214b700 1 -- 172.21.15.31:0/2643837816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f612c1008f0 msgr2=0x7f612c0fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f613214b700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f612c1008f0 0x7f612c0fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f613214b700 1 -- 172.21.15.31:0/2643837816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f612c1012b0 msgr2=0x7f612c0fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f613214b700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f612c1012b0 0x7f612c0fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f613214b700 1 -- 172.21.15.31:0/2643837816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c1a6df0 con 0x7f612c1068a0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f613214b700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 0x7f612c0fd690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6124007ee0 tx=0x7f612400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124004bb0 con 0x7f612c1068a0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6124004d50 con 0x7f612c1068a0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c1a70e0 con 0x7f612c1068a0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240058e0 con 0x7f612c1068a0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.275+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c1a7670 con 0x7f612c1068a0 2024-04-25T00:29:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.276+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c101fe0 con 0x7f612c1068a0 2024-04-25T00:29:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.277+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f612401a020 con 0x7f612c1068a0 2024-04-25T00:29:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.278+0000 7f6122ffd700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61180a2ca0 0x7f61180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.278+0000 7f613194a700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61180a2ca0 0x7f61180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.278+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f61240ce6b0 con 0x7f612c1068a0 2024-04-25T00:29:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.278+0000 7f613194a700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61180a2ca0 0x7f61180a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f611c00abd0 tx=0x7f611c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.282+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6124010070 con 0x7f612c1068a0 2024-04-25T00:29:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.485+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f612c062670 con 0x7f61180a2ca0 2024-04-25T00:29:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.493+0000 7f6122ffd700 1 -- 172.21.15.31:0/2643837816 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259168 (secure 0 0 0) 0x7f612c062670 con 0x7f61180a2ca0 2024-04-25T00:29:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61180a2ca0 msgr2=0x7f61180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61180a2ca0 0x7f61180a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f611c00abd0 tx=0x7f611c00a380 comp rx=0 tx=0).stop 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 msgr2=0x7f612c0fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 0x7f612c0fd690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6124007ee0 tx=0x7f612400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 shutdown_connections 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61180a2ca0 0x7f61180a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f612c1068a0 0x7f612c0fd690 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f612c1008f0 0x7f612c0fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 --2- 172.21.15.31:0/2643837816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f612c1012b0 0x7f612c0fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 >> 172.21.15.31:0/2643837816 conn(0x7f612c0777a0 msgr2=0x7f612c0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 shutdown_connections 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:47.496+0000 7f6133bae700 1 -- 172.21.15.31:0/2643837816 wait complete. 2024-04-25T00:29:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T00:29:48.287 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T00:29:47.323520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657579,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":919,"num_read_kb":574,"num_write":421,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365296,"kb_used_data":11952,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576528,"statfs":{"total":1151940427776,"available":1151566364672,"internally_reserved":0,"allocated":12238848,"data_stored":7542702,"data_compressed":177228,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.338457+0000","last_change":"2024-04-25T00:28:24.187326+0000","last_active":"2024-04-25T00:29:43.338457+0000","last_peered":"2024-04-25T00:29:43.338457+0000","last_clean":"2024-04-25T00:29:43.338457+0000","last_became_active":"2024-04-25T00:28:24.186536+0000","last_became_peered":"2024-04-25T00:28:24.186536+0000","last_unstale":"2024-04-25T00:29:43.338457+0000","last_undegraded":"2024-04-25T00:29:43.338457+0000","last_fullsized":"2024-04-25T00:29:43.338457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:25.035523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435544+0000","last_change":"2024-04-25T00:28:24.181763+0000","last_active":"2024-04-25T00:29:14.435544+0000","last_peered":"2024-04-25T00:29:14.435544+0000","last_clean":"2024-04-25T00:29:14.435544+0000","last_became_active":"2024-04-25T00:28:24.181437+0000","last_became_peered":"2024-04-25T00:28:24.181437+0000","last_unstale":"2024-04-25T00:29:14.435544+0000","last_undegraded":"2024-04-25T00:29:14.435544+0000","last_fullsized":"2024-04-25T00:29:14.435544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:54.019249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435253+0000","last_change":"2024-04-25T00:28:24.186648+0000","last_active":"2024-04-25T00:29:14.435253+0000","last_peered":"2024-04-25T00:29:14.435253+0000","last_clean":"2024-04-25T00:29:14.435253+0000","last_became_active":"2024-04-25T00:28:24.186303+0000","last_became_peered":"2024-04-25T00:28:24.186303+0000","last_unstale":"2024-04-25T00:29:14.435253+0000","last_undegraded":"2024-04-25T00:29:14.435253+0000","last_fullsized":"2024-04-25T00:29:14.435253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:14.521970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.628136+0000","last_change":"2024-04-25T00:28:24.179138+0000","last_active":"2024-04-25T00:29:14.628136+0000","last_peered":"2024-04-25T00:29:14.628136+0000","last_clean":"2024-04-25T00:29:14.628136+0000","last_became_active":"2024-04-25T00:28:24.178664+0000","last_became_peered":"2024-04-25T00:28:24.178664+0000","last_unstale":"2024-04-25T00:29:14.628136+0000","last_undegraded":"2024-04-25T00:29:14.628136+0000","last_fullsized":"2024-04-25T00:29:14.628136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:07.869662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433375+0000","last_change":"2024-04-25T00:28:24.174081+0000","last_active":"2024-04-25T00:29:14.433375+0000","last_peered":"2024-04-25T00:29:14.433375+0000","last_clean":"2024-04-25T00:29:14.433375+0000","last_became_active":"2024-04-25T00:28:24.173670+0000","last_became_peered":"2024-04-25T00:28:24.173670+0000","last_unstale":"2024-04-25T00:29:14.433375+0000","last_undegraded":"2024-04-25T00:29:14.433375+0000","last_fullsized":"2024-04-25T00:29:14.433375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:43.255598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431088+0000","last_change":"2024-04-25T00:28:24.174664+0000","last_active":"2024-04-25T00:29:14.431088+0000","last_peered":"2024-04-25T00:29:14.431088+0000","last_clean":"2024-04-25T00:29:14.431088+0000","last_became_active":"2024-04-25T00:28:24.174238+0000","last_became_peered":"2024-04-25T00:28:24.174238+0000","last_unstale":"2024-04-25T00:29:14.431088+0000","last_undegraded":"2024-04-25T00:29:14.431088+0000","last_fullsized":"2024-04-25T00:29:14.431088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:21.214976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.341448+0000","last_change":"2024-04-25T00:28:24.181123+0000","last_active":"2024-04-25T00:29:43.341448+0000","last_peered":"2024-04-25T00:29:43.341448+0000","last_clean":"2024-04-25T00:29:43.341448+0000","last_became_active":"2024-04-25T00:28:24.179516+0000","last_became_peered":"2024-04-25T00:28:24.179516+0000","last_unstale":"2024-04-25T00:29:43.341448+0000","last_undegraded":"2024-04-25T00:29:43.341448+0000","last_fullsized":"2024-04-25T00:29:43.341448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:11.822968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.437326+0000","last_change":"2024-04-25T00:29:14.436445+0000","last_active":"2024-04-25T00:29:14.437326+0000","last_peered":"2024-04-25T00:29:14.437326+0000","last_clean":"2024-04-25T00:29:14.437326+0000","last_became_active":"2024-04-25T00:29:14.436069+0000","last_became_peered":"2024-04-25T00:29:14.436069+0000","last_unstale":"2024-04-25T00:29:14.437326+0000","last_undegraded":"2024-04-25T00:29:14.437326+0000","last_fullsized":"2024-04-25T00:29:14.437326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:43.030641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664707+0000","last_change":"2024-04-25T00:28:24.170818+0000","last_active":"2024-04-25T00:29:14.664707+0000","last_peered":"2024-04-25T00:29:14.664707+0000","last_clean":"2024-04-25T00:29:14.664707+0000","last_became_active":"2024-04-25T00:28:24.170412+0000","last_became_peered":"2024-04-25T00:28:24.170412+0000","last_unstale":"2024-04-25T00:29:14.664707+0000","last_undegraded":"2024-04-25T00:29:14.664707+0000","last_fullsized":"2024-04-25T00:29:14.664707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:20.456317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431618+0000","last_change":"2024-04-25T00:28:24.182509+0000","last_active":"2024-04-25T00:29:14.431618+0000","last_peered":"2024-04-25T00:29:14.431618+0000","last_clean":"2024-04-25T00:29:14.431618+0000","last_became_active":"2024-04-25T00:28:24.182141+0000","last_became_peered":"2024-04-25T00:28:24.182141+0000","last_unstale":"2024-04-25T00:29:14.431618+0000","last_undegraded":"2024-04-25T00:29:14.431618+0000","last_fullsized":"2024-04-25T00:29:14.431618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:33.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.340199+0000","last_change":"2024-04-25T00:28:24.189265+0000","last_active":"2024-04-25T00:29:43.340199+0000","last_peered":"2024-04-25T00:29:43.340199+0000","last_clean":"2024-04-25T00:29:43.340199+0000","last_became_active":"2024-04-25T00:28:24.188829+0000","last_became_peered":"2024-04-25T00:28:24.188829+0000","last_unstale":"2024-04-25T00:29:43.340199+0000","last_undegraded":"2024-04-25T00:29:43.340199+0000","last_fullsized":"2024-04-25T00:29:43.340199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:33.930373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.437356+0000","last_change":"2024-04-25T00:29:14.436642+0000","last_active":"2024-04-25T00:29:14.437356+0000","last_peered":"2024-04-25T00:29:14.437356+0000","last_clean":"2024-04-25T00:29:14.437356+0000","last_became_active":"2024-04-25T00:29:14.436291+0000","last_became_peered":"2024-04-25T00:29:14.436291+0000","last_unstale":"2024-04-25T00:29:14.437356+0000","last_undegraded":"2024-04-25T00:29:14.437356+0000","last_fullsized":"2024-04-25T00:29:14.437356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:03.844556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.332113+0000","last_change":"2024-04-25T00:28:24.183769+0000","last_active":"2024-04-25T00:29:43.332113+0000","last_peered":"2024-04-25T00:29:43.332113+0000","last_clean":"2024-04-25T00:29:43.332113+0000","last_became_active":"2024-04-25T00:28:24.183432+0000","last_became_peered":"2024-04-25T00:28:24.183432+0000","last_unstale":"2024-04-25T00:29:43.332113+0000","last_undegraded":"2024-04-25T00:29:43.332113+0000","last_fullsized":"2024-04-25T00:29:43.332113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:27.656794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435041+0000","last_change":"2024-04-25T00:28:24.184364+0000","last_active":"2024-04-25T00:29:14.435041+0000","last_peered":"2024-04-25T00:29:14.435041+0000","last_clean":"2024-04-25T00:29:14.435041+0000","last_became_active":"2024-04-25T00:28:24.184038+0000","last_became_peered":"2024-04-25T00:28:24.184038+0000","last_unstale":"2024-04-25T00:29:14.435041+0000","last_undegraded":"2024-04-25T00:29:14.435041+0000","last_fullsized":"2024-04-25T00:29:14.435041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:50.495466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434953+0000","last_change":"2024-04-25T00:28:24.180236+0000","last_active":"2024-04-25T00:29:14.434953+0000","last_peered":"2024-04-25T00:29:14.434953+0000","last_clean":"2024-04-25T00:29:14.434953+0000","last_became_active":"2024-04-25T00:28:24.179929+0000","last_became_peered":"2024-04-25T00:28:24.179929+0000","last_unstale":"2024-04-25T00:29:14.434953+0000","last_undegraded":"2024-04-25T00:29:14.434953+0000","last_fullsized":"2024-04-25T00:29:14.434953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:29.858564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664086+0000","last_change":"2024-04-25T00:28:24.185605+0000","last_active":"2024-04-25T00:29:14.664086+0000","last_peered":"2024-04-25T00:29:14.664086+0000","last_clean":"2024-04-25T00:29:14.664086+0000","last_became_active":"2024-04-25T00:28:24.185270+0000","last_became_peered":"2024-04-25T00:28:24.185270+0000","last_unstale":"2024-04-25T00:29:14.664086+0000","last_undegraded":"2024-04-25T00:29:14.664086+0000","last_fullsized":"2024-04-25T00:29:14.664086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:58.798429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.338776+0000","last_change":"2024-04-25T00:28:24.187625+0000","last_active":"2024-04-25T00:29:43.338776+0000","last_peered":"2024-04-25T00:29:43.338776+0000","last_clean":"2024-04-25T00:29:43.338776+0000","last_became_active":"2024-04-25T00:28:24.186961+0000","last_became_peered":"2024-04-25T00:28:24.186961+0000","last_unstale":"2024-04-25T00:29:43.338776+0000","last_undegraded":"2024-04-25T00:29:43.338776+0000","last_fullsized":"2024-04-25T00:29:43.338776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:26.099099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429841+0000","last_change":"2024-04-25T00:28:24.177342+0000","last_active":"2024-04-25T00:29:14.429841+0000","last_peered":"2024-04-25T00:29:14.429841+0000","last_clean":"2024-04-25T00:29:14.429841+0000","last_became_active":"2024-04-25T00:28:24.176763+0000","last_became_peered":"2024-04-25T00:28:24.176763+0000","last_unstale":"2024-04-25T00:29:14.429841+0000","last_undegraded":"2024-04-25T00:29:14.429841+0000","last_fullsized":"2024-04-25T00:29:14.429841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:47.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.325308+0000","last_change":"2024-04-25T00:28:24.186069+0000","last_active":"2024-04-25T00:29:43.325308+0000","last_peered":"2024-04-25T00:29:43.325308+0000","last_clean":"2024-04-25T00:29:43.325308+0000","last_became_active":"2024-04-25T00:28:24.185713+0000","last_became_peered":"2024-04-25T00:28:24.185713+0000","last_unstale":"2024-04-25T00:29:43.325308+0000","last_undegraded":"2024-04-25T00:29:43.325308+0000","last_fullsized":"2024-04-25T00:29:43.325308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:08.704071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":106,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:41.284730+0000","last_change":"2024-04-25T00:28:23.402187+0000","last_active":"2024-04-25T00:29:41.284730+0000","last_peered":"2024-04-25T00:29:41.284730+0000","last_clean":"2024-04-25T00:29:41.284730+0000","last_became_active":"2024-04-25T00:28:23.401868+0000","last_became_peered":"2024-04-25T00:28:23.401868+0000","last_unstale":"2024-04-25T00:29:41.284730+0000","last_undegraded":"2024-04-25T00:29:41.284730+0000","last_fullsized":"2024-04-25T00:29:41.284730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:44.022628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.335480+0000","last_change":"2024-04-25T00:28:24.180731+0000","last_active":"2024-04-25T00:29:43.335480+0000","last_peered":"2024-04-25T00:29:43.335480+0000","last_clean":"2024-04-25T00:29:43.335480+0000","last_became_active":"2024-04-25T00:28:24.180274+0000","last_became_peered":"2024-04-25T00:28:24.180274+0000","last_unstale":"2024-04-25T00:29:43.335480+0000","last_undegraded":"2024-04-25T00:29:43.335480+0000","last_fullsized":"2024-04-25T00:29:43.335480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:46.102326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439378+0000","last_change":"2024-04-25T00:28:23.405683+0000","last_active":"2024-04-25T00:29:14.439378+0000","last_peered":"2024-04-25T00:29:14.439378+0000","last_clean":"2024-04-25T00:29:14.439378+0000","last_became_active":"2024-04-25T00:28:23.405305+0000","last_became_peered":"2024-04-25T00:28:23.405305+0000","last_unstale":"2024-04-25T00:29:14.439378+0000","last_undegraded":"2024-04-25T00:29:14.439378+0000","last_fullsized":"2024-04-25T00:29:14.439378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:39.782047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435803+0000","last_change":"2024-04-25T00:28:24.187565+0000","last_active":"2024-04-25T00:29:14.435803+0000","last_peered":"2024-04-25T00:29:14.435803+0000","last_clean":"2024-04-25T00:29:14.435803+0000","last_became_active":"2024-04-25T00:28:24.186397+0000","last_became_peered":"2024-04-25T00:28:24.186397+0000","last_unstale":"2024-04-25T00:29:14.435803+0000","last_undegraded":"2024-04-25T00:29:14.435803+0000","last_fullsized":"2024-04-25T00:29:14.435803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:36.750031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433092+0000","last_change":"2024-04-25T00:28:23.398726+0000","last_active":"2024-04-25T00:29:14.433092+0000","last_peered":"2024-04-25T00:29:14.433092+0000","last_clean":"2024-04-25T00:29:14.433092+0000","last_became_active":"2024-04-25T00:28:23.398370+0000","last_became_peered":"2024-04-25T00:28:23.398370+0000","last_unstale":"2024-04-25T00:29:14.433092+0000","last_undegraded":"2024-04-25T00:29:14.433092+0000","last_fullsized":"2024-04-25T00:29:14.433092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:19.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664297+0000","last_change":"2024-04-25T00:28:24.181424+0000","last_active":"2024-04-25T00:29:14.664297+0000","last_peered":"2024-04-25T00:29:14.664297+0000","last_clean":"2024-04-25T00:29:14.664297+0000","last_became_active":"2024-04-25T00:28:24.179077+0000","last_became_peered":"2024-04-25T00:28:24.179077+0000","last_unstale":"2024-04-25T00:29:14.664297+0000","last_undegraded":"2024-04-25T00:29:14.664297+0000","last_fullsized":"2024-04-25T00:29:14.664297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:17.926451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432153+0000","last_change":"2024-04-25T00:28:23.402419+0000","last_active":"2024-04-25T00:29:14.432153+0000","last_peered":"2024-04-25T00:29:14.432153+0000","last_clean":"2024-04-25T00:29:14.432153+0000","last_became_active":"2024-04-25T00:28:23.402070+0000","last_became_peered":"2024-04-25T00:28:23.402070+0000","last_unstale":"2024-04-25T00:29:14.432153+0000","last_undegraded":"2024-04-25T00:29:14.432153+0000","last_fullsized":"2024-04-25T00:29:14.432153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:08.799577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.342451+0000","last_change":"2024-04-25T00:28:24.178856+0000","last_active":"2024-04-25T00:29:43.342451+0000","last_peered":"2024-04-25T00:29:43.342451+0000","last_clean":"2024-04-25T00:29:43.342451+0000","last_became_active":"2024-04-25T00:28:24.178469+0000","last_became_peered":"2024-04-25T00:28:24.178469+0000","last_unstale":"2024-04-25T00:29:43.342451+0000","last_undegraded":"2024-04-25T00:29:43.342451+0000","last_fullsized":"2024-04-25T00:29:43.342451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:28.211603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":109,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:45.274656+0000","last_change":"2024-04-25T00:28:23.400250+0000","last_active":"2024-04-25T00:29:45.274656+0000","last_peered":"2024-04-25T00:29:45.274656+0000","last_clean":"2024-04-25T00:29:45.274656+0000","last_became_active":"2024-04-25T00:28:23.399874+0000","last_became_peered":"2024-04-25T00:28:23.399874+0000","last_unstale":"2024-04-25T00:29:45.274656+0000","last_undegraded":"2024-04-25T00:29:45.274656+0000","last_fullsized":"2024-04-25T00:29:45.274656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:08.332441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432947+0000","last_change":"2024-04-25T00:28:24.170176+0000","last_active":"2024-04-25T00:29:14.432947+0000","last_peered":"2024-04-25T00:29:14.432947+0000","last_clean":"2024-04-25T00:29:14.432947+0000","last_became_active":"2024-04-25T00:28:24.169493+0000","last_became_peered":"2024-04-25T00:28:24.169493+0000","last_unstale":"2024-04-25T00:29:14.432947+0000","last_undegraded":"2024-04-25T00:29:14.432947+0000","last_fullsized":"2024-04-25T00:29:14.432947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:43.855777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.670996+0000","last_change":"2024-04-25T00:29:14.670996+0000","last_active":"2024-04-25T00:29:14.670996+0000","last_peered":"2024-04-25T00:29:14.670996+0000","last_clean":"2024-04-25T00:29:14.670996+0000","last_became_active":"2024-04-25T00:29:14.665864+0000","last_became_peered":"2024-04-25T00:29:14.665864+0000","last_unstale":"2024-04-25T00:29:14.670996+0000","last_undegraded":"2024-04-25T00:29:14.670996+0000","last_fullsized":"2024-04-25T00:29:14.670996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:18.011605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.340549+0000","last_change":"2024-04-25T00:29:14.629508+0000","last_active":"2024-04-25T00:29:43.340549+0000","last_peered":"2024-04-25T00:29:43.340549+0000","last_clean":"2024-04-25T00:29:43.340549+0000","last_became_active":"2024-04-25T00:29:14.629037+0000","last_became_peered":"2024-04-25T00:29:14.629037+0000","last_unstale":"2024-04-25T00:29:43.340549+0000","last_undegraded":"2024-04-25T00:29:43.340549+0000","last_fullsized":"2024-04-25T00:29:43.340549+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:47.354462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433092+0000","last_change":"2024-04-25T00:28:23.397855+0000","last_active":"2024-04-25T00:29:14.433092+0000","last_peered":"2024-04-25T00:29:14.433092+0000","last_clean":"2024-04-25T00:29:14.433092+0000","last_became_active":"2024-04-25T00:28:23.397465+0000","last_became_peered":"2024-04-25T00:28:23.397465+0000","last_unstale":"2024-04-25T00:29:14.433092+0000","last_undegraded":"2024-04-25T00:29:14.433092+0000","last_fullsized":"2024-04-25T00:29:14.433092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.357397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.317289+0000","last_change":"2024-04-25T00:28:24.172890+0000","last_active":"2024-04-25T00:29:43.317289+0000","last_peered":"2024-04-25T00:29:43.317289+0000","last_clean":"2024-04-25T00:29:43.317289+0000","last_became_active":"2024-04-25T00:28:24.172250+0000","last_became_peered":"2024-04-25T00:28:24.172250+0000","last_unstale":"2024-04-25T00:29:43.317289+0000","last_undegraded":"2024-04-25T00:29:43.317289+0000","last_fullsized":"2024-04-25T00:29:43.317289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:34.372811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434906+0000","last_change":"2024-04-25T00:28:23.401140+0000","last_active":"2024-04-25T00:29:14.434906+0000","last_peered":"2024-04-25T00:29:14.434906+0000","last_clean":"2024-04-25T00:29:14.434906+0000","last_became_active":"2024-04-25T00:28:23.400804+0000","last_became_peered":"2024-04-25T00:28:23.400804+0000","last_unstale":"2024-04-25T00:29:14.434906+0000","last_undegraded":"2024-04-25T00:29:14.434906+0000","last_fullsized":"2024-04-25T00:29:14.434906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:00.044141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429781+0000","last_change":"2024-04-25T00:28:24.175596+0000","last_active":"2024-04-25T00:29:14.429781+0000","last_peered":"2024-04-25T00:29:14.429781+0000","last_clean":"2024-04-25T00:29:14.429781+0000","last_became_active":"2024-04-25T00:28:24.175085+0000","last_became_peered":"2024-04-25T00:28:24.175085+0000","last_unstale":"2024-04-25T00:29:14.429781+0000","last_undegraded":"2024-04-25T00:29:14.429781+0000","last_fullsized":"2024-04-25T00:29:14.429781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:05.801995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:42.268990+0000","last_change":"2024-04-25T00:28:23.399427+0000","last_active":"2024-04-25T00:29:42.268990+0000","last_peered":"2024-04-25T00:29:42.268990+0000","last_clean":"2024-04-25T00:29:42.268990+0000","last_became_active":"2024-04-25T00:28:23.399067+0000","last_became_peered":"2024-04-25T00:28:23.399067+0000","last_unstale":"2024-04-25T00:29:42.268990+0000","last_undegraded":"2024-04-25T00:29:42.268990+0000","last_fullsized":"2024-04-25T00:29:42.268990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:06.853292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664998+0000","last_change":"2024-04-25T00:28:24.177367+0000","last_active":"2024-04-25T00:29:14.664998+0000","last_peered":"2024-04-25T00:29:14.664998+0000","last_clean":"2024-04-25T00:29:14.664998+0000","last_became_active":"2024-04-25T00:28:24.176890+0000","last_became_peered":"2024-04-25T00:28:24.176890+0000","last_unstale":"2024-04-25T00:29:14.664998+0000","last_undegraded":"2024-04-25T00:29:14.664998+0000","last_fullsized":"2024-04-25T00:29:14.664998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:27.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664639+0000","last_change":"2024-04-25T00:28:23.409705+0000","last_active":"2024-04-25T00:29:14.664639+0000","last_peered":"2024-04-25T00:29:14.664639+0000","last_clean":"2024-04-25T00:29:14.664639+0000","last_became_active":"2024-04-25T00:28:23.408596+0000","last_became_peered":"2024-04-25T00:28:23.408596+0000","last_unstale":"2024-04-25T00:29:14.664639+0000","last_undegraded":"2024-04-25T00:29:14.664639+0000","last_fullsized":"2024-04-25T00:29:14.664639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:25.684497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435704+0000","last_change":"2024-04-25T00:28:24.179361+0000","last_active":"2024-04-25T00:29:14.435704+0000","last_peered":"2024-04-25T00:29:14.435704+0000","last_clean":"2024-04-25T00:29:14.435704+0000","last_became_active":"2024-04-25T00:28:24.179019+0000","last_became_peered":"2024-04-25T00:28:24.179019+0000","last_unstale":"2024-04-25T00:29:14.435704+0000","last_undegraded":"2024-04-25T00:29:14.435704+0000","last_fullsized":"2024-04-25T00:29:14.435704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:08.543283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435090+0000","last_change":"2024-04-25T00:28:23.404051+0000","last_active":"2024-04-25T00:29:14.435090+0000","last_peered":"2024-04-25T00:29:14.435090+0000","last_clean":"2024-04-25T00:29:14.435090+0000","last_became_active":"2024-04-25T00:28:23.403728+0000","last_became_peered":"2024-04-25T00:28:23.403728+0000","last_unstale":"2024-04-25T00:29:14.435090+0000","last_undegraded":"2024-04-25T00:29:14.435090+0000","last_fullsized":"2024-04-25T00:29:14.435090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:27.403464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664511+0000","last_change":"2024-04-25T00:28:24.178506+0000","last_active":"2024-04-25T00:29:14.664511+0000","last_peered":"2024-04-25T00:29:14.664511+0000","last_clean":"2024-04-25T00:29:14.664511+0000","last_became_active":"2024-04-25T00:28:24.177053+0000","last_became_peered":"2024-04-25T00:28:24.177053+0000","last_unstale":"2024-04-25T00:29:14.664511+0000","last_undegraded":"2024-04-25T00:29:14.664511+0000","last_fullsized":"2024-04-25T00:29:14.664511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:04.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:42.285951+0000","last_change":"2024-04-25T00:28:23.400563+0000","last_active":"2024-04-25T00:29:42.285951+0000","last_peered":"2024-04-25T00:29:42.285951+0000","last_clean":"2024-04-25T00:29:42.285951+0000","last_became_active":"2024-04-25T00:28:23.399982+0000","last_became_peered":"2024-04-25T00:28:23.399982+0000","last_unstale":"2024-04-25T00:29:42.285951+0000","last_undegraded":"2024-04-25T00:29:42.285951+0000","last_fullsized":"2024-04-25T00:29:42.285951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:01.652027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":136,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433951+0000","last_change":"2024-04-25T00:28:24.179687+0000","last_active":"2024-04-25T00:29:14.433951+0000","last_peered":"2024-04-25T00:29:14.433951+0000","last_clean":"2024-04-25T00:29:14.433951+0000","last_became_active":"2024-04-25T00:28:24.179345+0000","last_became_peered":"2024-04-25T00:28:24.179345+0000","last_unstale":"2024-04-25T00:29:14.433951+0000","last_undegraded":"2024-04-25T00:29:14.433951+0000","last_fullsized":"2024-04-25T00:29:14.433951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:49.807417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432873+0000","last_change":"2024-04-25T00:28:23.401353+0000","last_active":"2024-04-25T00:29:14.432873+0000","last_peered":"2024-04-25T00:29:14.432873+0000","last_clean":"2024-04-25T00:29:14.432873+0000","last_became_active":"2024-04-25T00:28:23.400985+0000","last_became_peered":"2024-04-25T00:28:23.400985+0000","last_unstale":"2024-04-25T00:29:14.432873+0000","last_undegraded":"2024-04-25T00:29:14.432873+0000","last_fullsized":"2024-04-25T00:29:14.432873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.277514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.336727+0000","last_change":"2024-04-25T00:28:24.178796+0000","last_active":"2024-04-25T00:29:43.336727+0000","last_peered":"2024-04-25T00:29:43.336727+0000","last_clean":"2024-04-25T00:29:43.336727+0000","last_became_active":"2024-04-25T00:28:24.178457+0000","last_became_peered":"2024-04-25T00:28:24.178457+0000","last_unstale":"2024-04-25T00:29:43.336727+0000","last_undegraded":"2024-04-25T00:29:43.336727+0000","last_fullsized":"2024-04-25T00:29:43.336727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:48.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434395+0000","last_change":"2024-04-25T00:28:23.406046+0000","last_active":"2024-04-25T00:29:14.434395+0000","last_peered":"2024-04-25T00:29:14.434395+0000","last_clean":"2024-04-25T00:29:14.434395+0000","last_became_active":"2024-04-25T00:28:23.405686+0000","last_became_peered":"2024-04-25T00:28:23.405686+0000","last_unstale":"2024-04-25T00:29:14.434395+0000","last_undegraded":"2024-04-25T00:29:14.434395+0000","last_fullsized":"2024-04-25T00:29:14.434395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:30.551884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.336704+0000","last_change":"2024-04-25T00:28:24.188251+0000","last_active":"2024-04-25T00:29:43.336704+0000","last_peered":"2024-04-25T00:29:43.336704+0000","last_clean":"2024-04-25T00:29:43.336704+0000","last_became_active":"2024-04-25T00:28:24.187876+0000","last_became_peered":"2024-04-25T00:28:24.187876+0000","last_unstale":"2024-04-25T00:29:43.336704+0000","last_undegraded":"2024-04-25T00:29:43.336704+0000","last_fullsized":"2024-04-25T00:29:43.336704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:18.269074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:41.267345+0000","last_change":"2024-04-25T00:28:23.407240+0000","last_active":"2024-04-25T00:29:41.267345+0000","last_peered":"2024-04-25T00:29:41.267345+0000","last_clean":"2024-04-25T00:29:41.267345+0000","last_became_active":"2024-04-25T00:28:23.406794+0000","last_became_peered":"2024-04-25T00:28:23.406794+0000","last_unstale":"2024-04-25T00:29:41.267345+0000","last_undegraded":"2024-04-25T00:29:41.267345+0000","last_fullsized":"2024-04-25T00:29:41.267345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:35.861733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.341872+0000","last_change":"2024-04-25T00:28:24.183784+0000","last_active":"2024-04-25T00:29:43.341872+0000","last_peered":"2024-04-25T00:29:43.341872+0000","last_clean":"2024-04-25T00:29:43.341872+0000","last_became_active":"2024-04-25T00:28:24.183417+0000","last_became_peered":"2024-04-25T00:28:24.183417+0000","last_unstale":"2024-04-25T00:29:43.341872+0000","last_undegraded":"2024-04-25T00:29:43.341872+0000","last_fullsized":"2024-04-25T00:29:43.341872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:43.182050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.642117+0000","last_change":"2024-04-25T00:29:14.642117+0000","last_active":"2024-04-25T00:29:14.642117+0000","last_peered":"2024-04-25T00:29:14.642117+0000","last_clean":"2024-04-25T00:29:14.642117+0000","last_became_active":"2024-04-25T00:29:14.629616+0000","last_became_peered":"2024-04-25T00:29:14.629616+0000","last_unstale":"2024-04-25T00:29:14.642117+0000","last_undegraded":"2024-04-25T00:29:14.642117+0000","last_fullsized":"2024-04-25T00:29:14.642117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:50.342238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.338554+0000","last_change":"2024-04-25T00:28:24.177898+0000","last_active":"2024-04-25T00:29:43.338554+0000","last_peered":"2024-04-25T00:29:43.338554+0000","last_clean":"2024-04-25T00:29:43.338554+0000","last_became_active":"2024-04-25T00:28:24.177578+0000","last_became_peered":"2024-04-25T00:28:24.177578+0000","last_unstale":"2024-04-25T00:29:43.338554+0000","last_undegraded":"2024-04-25T00:29:43.338554+0000","last_fullsized":"2024-04-25T00:29:43.338554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:36.294161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432401+0000","last_change":"2024-04-25T00:28:23.399570+0000","last_active":"2024-04-25T00:29:14.432401+0000","last_peered":"2024-04-25T00:29:14.432401+0000","last_clean":"2024-04-25T00:29:14.432401+0000","last_became_active":"2024-04-25T00:28:23.399230+0000","last_became_peered":"2024-04-25T00:28:23.399230+0000","last_unstale":"2024-04-25T00:29:14.432401+0000","last_undegraded":"2024-04-25T00:29:14.432401+0000","last_fullsized":"2024-04-25T00:29:14.432401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:46.850590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.324901+0000","last_change":"2024-04-25T00:28:24.186113+0000","last_active":"2024-04-25T00:29:43.324901+0000","last_peered":"2024-04-25T00:29:43.324901+0000","last_clean":"2024-04-25T00:29:43.324901+0000","last_became_active":"2024-04-25T00:28:24.185743+0000","last_became_peered":"2024-04-25T00:28:24.185743+0000","last_unstale":"2024-04-25T00:29:43.324901+0000","last_undegraded":"2024-04-25T00:29:43.324901+0000","last_fullsized":"2024-04-25T00:29:43.324901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:10.973285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429923+0000","last_change":"2024-04-25T00:28:23.396054+0000","last_active":"2024-04-25T00:29:14.429923+0000","last_peered":"2024-04-25T00:29:14.429923+0000","last_clean":"2024-04-25T00:29:14.429923+0000","last_became_active":"2024-04-25T00:28:23.395707+0000","last_became_peered":"2024-04-25T00:28:23.395707+0000","last_unstale":"2024-04-25T00:29:14.429923+0000","last_undegraded":"2024-04-25T00:29:14.429923+0000","last_fullsized":"2024-04-25T00:29:14.429923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:20.169476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431543+0000","last_change":"2024-04-25T00:28:24.185655+0000","last_active":"2024-04-25T00:29:14.431543+0000","last_peered":"2024-04-25T00:29:14.431543+0000","last_clean":"2024-04-25T00:29:14.431543+0000","last_became_active":"2024-04-25T00:28:24.185123+0000","last_became_peered":"2024-04-25T00:28:24.185123+0000","last_unstale":"2024-04-25T00:29:14.431543+0000","last_undegraded":"2024-04-25T00:29:14.431543+0000","last_fullsized":"2024-04-25T00:29:14.431543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.845762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.287606+0000","last_change":"2024-04-25T00:28:23.410964+0000","last_active":"2024-04-25T00:29:43.287606+0000","last_peered":"2024-04-25T00:29:43.287606+0000","last_clean":"2024-04-25T00:29:43.287606+0000","last_became_active":"2024-04-25T00:28:23.410602+0000","last_became_peered":"2024-04-25T00:28:23.410602+0000","last_unstale":"2024-04-25T00:29:43.287606+0000","last_undegraded":"2024-04-25T00:29:43.287606+0000","last_fullsized":"2024-04-25T00:29:43.287606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:24.495790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431458+0000","last_change":"2024-04-25T00:28:24.184710+0000","last_active":"2024-04-25T00:29:14.431458+0000","last_peered":"2024-04-25T00:29:14.431458+0000","last_clean":"2024-04-25T00:29:14.431458+0000","last_became_active":"2024-04-25T00:28:24.184354+0000","last_became_peered":"2024-04-25T00:28:24.184354+0000","last_unstale":"2024-04-25T00:29:14.431458+0000","last_undegraded":"2024-04-25T00:29:14.431458+0000","last_fullsized":"2024-04-25T00:29:14.431458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:47.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.333061+0000","last_change":"2024-04-25T00:29:14.438916+0000","last_active":"2024-04-25T00:29:43.333061+0000","last_peered":"2024-04-25T00:29:43.333061+0000","last_clean":"2024-04-25T00:29:43.333061+0000","last_became_active":"2024-04-25T00:29:14.438376+0000","last_became_peered":"2024-04-25T00:29:14.438376+0000","last_unstale":"2024-04-25T00:29:43.333061+0000","last_undegraded":"2024-04-25T00:29:43.333061+0000","last_fullsized":"2024-04-25T00:29:43.333061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:58.938866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:42.268157+0000","last_change":"2024-04-25T00:28:23.401437+0000","last_active":"2024-04-25T00:29:42.268157+0000","last_peered":"2024-04-25T00:29:42.268157+0000","last_clean":"2024-04-25T00:29:42.268157+0000","last_became_active":"2024-04-25T00:28:23.401091+0000","last_became_peered":"2024-04-25T00:28:23.401091+0000","last_unstale":"2024-04-25T00:29:42.268157+0000","last_undegraded":"2024-04-25T00:29:42.268157+0000","last_fullsized":"2024-04-25T00:29:42.268157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:58.748461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.335928+0000","last_change":"2024-04-25T00:28:24.166224+0000","last_active":"2024-04-25T00:29:43.335928+0000","last_peered":"2024-04-25T00:29:43.335928+0000","last_clean":"2024-04-25T00:29:43.335928+0000","last_became_active":"2024-04-25T00:28:24.165626+0000","last_became_peered":"2024-04-25T00:28:24.165626+0000","last_unstale":"2024-04-25T00:29:43.335928+0000","last_undegraded":"2024-04-25T00:29:43.335928+0000","last_fullsized":"2024-04-25T00:29:43.335928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:04.956168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.628121+0000","last_change":"2024-04-25T00:28:23.408551+0000","last_active":"2024-04-25T00:29:14.628121+0000","last_peered":"2024-04-25T00:29:14.628121+0000","last_clean":"2024-04-25T00:29:14.628121+0000","last_became_active":"2024-04-25T00:28:23.408137+0000","last_became_peered":"2024-04-25T00:28:23.408137+0000","last_unstale":"2024-04-25T00:29:14.628121+0000","last_undegraded":"2024-04-25T00:29:14.628121+0000","last_fullsized":"2024-04-25T00:29:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:53.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.663940+0000","last_change":"2024-04-25T00:27:10.587192+0000","last_active":"2024-04-25T00:29:14.663940+0000","last_peered":"2024-04-25T00:29:14.663940+0000","last_clean":"2024-04-25T00:29:14.663940+0000","last_became_active":"2024-04-25T00:27:10.586481+0000","last_became_peered":"2024-04-25T00:27:10.586481+0000","last_unstale":"2024-04-25T00:29:14.663940+0000","last_undegraded":"2024-04-25T00:29:14.663940+0000","last_fullsized":"2024-04-25T00:29:14.663940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:23:42.305887+0000","last_clean_scrub_stamp":"2024-04-25T00:23:42.305887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:44.337443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.326711+0000","last_change":"2024-04-25T00:29:14.437268+0000","last_active":"2024-04-25T00:29:43.326711+0000","last_peered":"2024-04-25T00:29:43.326711+0000","last_clean":"2024-04-25T00:29:43.326711+0000","last_became_active":"2024-04-25T00:29:14.436284+0000","last_became_peered":"2024-04-25T00:29:14.436284+0000","last_unstale":"2024-04-25T00:29:43.326711+0000","last_undegraded":"2024-04-25T00:29:43.326711+0000","last_fullsized":"2024-04-25T00:29:43.326711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:42.576164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664050+0000","last_change":"2024-04-25T00:28:23.404002+0000","last_active":"2024-04-25T00:29:14.664050+0000","last_peered":"2024-04-25T00:29:14.664050+0000","last_clean":"2024-04-25T00:29:14.664050+0000","last_became_active":"2024-04-25T00:28:23.403659+0000","last_became_peered":"2024-04-25T00:28:23.403659+0000","last_unstale":"2024-04-25T00:29:14.664050+0000","last_undegraded":"2024-04-25T00:29:14.664050+0000","last_fullsized":"2024-04-25T00:29:14.664050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:26.994242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.432875+0000","last_change":"2024-04-25T00:28:23.405525+0000","last_active":"2024-04-25T00:29:14.432875+0000","last_peered":"2024-04-25T00:29:14.432875+0000","last_clean":"2024-04-25T00:29:14.432875+0000","last_became_active":"2024-04-25T00:28:23.405228+0000","last_became_peered":"2024-04-25T00:28:23.405228+0000","last_unstale":"2024-04-25T00:29:14.432875+0000","last_undegraded":"2024-04-25T00:29:14.432875+0000","last_fullsized":"2024-04-25T00:29:14.432875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:00.819210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.326806+0000","last_change":"2024-04-25T00:28:24.182350+0000","last_active":"2024-04-25T00:29:43.326806+0000","last_peered":"2024-04-25T00:29:43.326806+0000","last_clean":"2024-04-25T00:29:43.326806+0000","last_became_active":"2024-04-25T00:28:24.182002+0000","last_became_peered":"2024-04-25T00:28:24.182002+0000","last_unstale":"2024-04-25T00:29:43.326806+0000","last_undegraded":"2024-04-25T00:29:43.326806+0000","last_fullsized":"2024-04-25T00:29:43.326806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:55.891046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.429957+0000","last_change":"2024-04-25T00:28:23.405357+0000","last_active":"2024-04-25T00:29:14.429957+0000","last_peered":"2024-04-25T00:29:14.429957+0000","last_clean":"2024-04-25T00:29:14.429957+0000","last_became_active":"2024-04-25T00:28:23.405031+0000","last_became_peered":"2024-04-25T00:28:23.405031+0000","last_unstale":"2024-04-25T00:29:14.429957+0000","last_undegraded":"2024-04-25T00:29:14.429957+0000","last_fullsized":"2024-04-25T00:29:14.429957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:10.683462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.334846+0000","last_change":"2024-04-25T00:28:24.177859+0000","last_active":"2024-04-25T00:29:43.334846+0000","last_peered":"2024-04-25T00:29:43.334846+0000","last_clean":"2024-04-25T00:29:43.334846+0000","last_became_active":"2024-04-25T00:28:24.177533+0000","last_became_peered":"2024-04-25T00:28:24.177533+0000","last_unstale":"2024-04-25T00:29:43.334846+0000","last_undegraded":"2024-04-25T00:29:43.334846+0000","last_fullsized":"2024-04-25T00:29:43.334846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:46.964934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.628107+0000","last_change":"2024-04-25T00:28:23.409487+0000","last_active":"2024-04-25T00:29:14.628107+0000","last_peered":"2024-04-25T00:29:14.628107+0000","last_clean":"2024-04-25T00:29:14.628107+0000","last_became_active":"2024-04-25T00:28:23.409146+0000","last_became_peered":"2024-04-25T00:28:23.409146+0000","last_unstale":"2024-04-25T00:29:14.628107+0000","last_undegraded":"2024-04-25T00:29:14.628107+0000","last_fullsized":"2024-04-25T00:29:14.628107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:28.591409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435503+0000","last_change":"2024-04-25T00:28:24.185083+0000","last_active":"2024-04-25T00:29:14.435503+0000","last_peered":"2024-04-25T00:29:14.435503+0000","last_clean":"2024-04-25T00:29:14.435503+0000","last_became_active":"2024-04-25T00:28:24.184752+0000","last_became_peered":"2024-04-25T00:28:24.184752+0000","last_unstale":"2024-04-25T00:29:14.435503+0000","last_undegraded":"2024-04-25T00:29:14.435503+0000","last_fullsized":"2024-04-25T00:29:14.435503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:08.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664144+0000","last_change":"2024-04-25T00:28:23.405662+0000","last_active":"2024-04-25T00:29:14.664144+0000","last_peered":"2024-04-25T00:29:14.664144+0000","last_clean":"2024-04-25T00:29:14.664144+0000","last_became_active":"2024-04-25T00:28:23.404949+0000","last_became_peered":"2024-04-25T00:28:23.404949+0000","last_unstale":"2024-04-25T00:29:14.664144+0000","last_undegraded":"2024-04-25T00:29:14.664144+0000","last_fullsized":"2024-04-25T00:29:14.664144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:01.989041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.431610+0000","last_change":"2024-04-25T00:28:24.181404+0000","last_active":"2024-04-25T00:29:14.431610+0000","last_peered":"2024-04-25T00:29:14.431610+0000","last_clean":"2024-04-25T00:29:14.431610+0000","last_became_active":"2024-04-25T00:28:24.180399+0000","last_became_peered":"2024-04-25T00:28:24.180399+0000","last_unstale":"2024-04-25T00:29:14.431610+0000","last_undegraded":"2024-04-25T00:29:14.431610+0000","last_fullsized":"2024-04-25T00:29:14.431610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:41.955204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.439444+0000","last_change":"2024-04-25T00:28:23.405288+0000","last_active":"2024-04-25T00:29:14.439444+0000","last_peered":"2024-04-25T00:29:14.439444+0000","last_clean":"2024-04-25T00:29:14.439444+0000","last_became_active":"2024-04-25T00:28:23.404547+0000","last_became_peered":"2024-04-25T00:28:23.404547+0000","last_unstale":"2024-04-25T00:29:14.439444+0000","last_undegraded":"2024-04-25T00:29:14.439444+0000","last_fullsized":"2024-04-25T00:29:14.439444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:45.180422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.327489+0000","last_change":"2024-04-25T00:28:24.176840+0000","last_active":"2024-04-25T00:29:43.327489+0000","last_peered":"2024-04-25T00:29:43.327489+0000","last_clean":"2024-04-25T00:29:43.327489+0000","last_became_active":"2024-04-25T00:28:24.176478+0000","last_became_peered":"2024-04-25T00:28:24.176478+0000","last_unstale":"2024-04-25T00:29:43.327489+0000","last_undegraded":"2024-04-25T00:29:43.327489+0000","last_fullsized":"2024-04-25T00:29:43.327489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:50.057929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.447278+0000","last_change":"2024-04-25T00:29:14.447278+0000","last_active":"2024-04-25T00:29:14.447278+0000","last_peered":"2024-04-25T00:29:14.447278+0000","last_clean":"2024-04-25T00:29:14.447278+0000","last_became_active":"2024-04-25T00:29:14.440971+0000","last_became_peered":"2024-04-25T00:29:14.440971+0000","last_unstale":"2024-04-25T00:29:14.447278+0000","last_undegraded":"2024-04-25T00:29:14.447278+0000","last_fullsized":"2024-04-25T00:29:14.447278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.256673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.334456+0000","last_change":"2024-04-25T00:28:24.176402+0000","last_active":"2024-04-25T00:29:43.334456+0000","last_peered":"2024-04-25T00:29:43.334456+0000","last_clean":"2024-04-25T00:29:43.334456+0000","last_became_active":"2024-04-25T00:28:24.175822+0000","last_became_peered":"2024-04-25T00:28:24.175822+0000","last_unstale":"2024-04-25T00:29:43.334456+0000","last_undegraded":"2024-04-25T00:29:43.334456+0000","last_fullsized":"2024-04-25T00:29:43.334456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:49.952116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433259+0000","last_change":"2024-04-25T00:28:23.401158+0000","last_active":"2024-04-25T00:29:14.433259+0000","last_peered":"2024-04-25T00:29:14.433259+0000","last_clean":"2024-04-25T00:29:14.433259+0000","last_became_active":"2024-04-25T00:28:23.400819+0000","last_became_peered":"2024-04-25T00:28:23.400819+0000","last_unstale":"2024-04-25T00:29:14.433259+0000","last_undegraded":"2024-04-25T00:29:14.433259+0000","last_fullsized":"2024-04-25T00:29:14.433259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.209636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.339876+0000","last_change":"2024-04-25T00:28:24.189784+0000","last_active":"2024-04-25T00:29:43.339876+0000","last_peered":"2024-04-25T00:29:43.339876+0000","last_clean":"2024-04-25T00:29:43.339876+0000","last_became_active":"2024-04-25T00:28:24.188161+0000","last_became_peered":"2024-04-25T00:28:24.188161+0000","last_unstale":"2024-04-25T00:29:43.339876+0000","last_undegraded":"2024-04-25T00:29:43.339876+0000","last_fullsized":"2024-04-25T00:29:43.339876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:06.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":103,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:45.293543+0000","last_change":"2024-04-25T00:28:23.410684+0000","last_active":"2024-04-25T00:29:45.293543+0000","last_peered":"2024-04-25T00:29:45.293543+0000","last_clean":"2024-04-25T00:29:45.293543+0000","last_became_active":"2024-04-25T00:28:23.410345+0000","last_became_peered":"2024-04-25T00:28:23.410345+0000","last_unstale":"2024-04-25T00:29:45.293543+0000","last_undegraded":"2024-04-25T00:29:45.293543+0000","last_fullsized":"2024-04-25T00:29:45.293543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.340552+0000","last_change":"2024-04-25T00:28:24.189170+0000","last_active":"2024-04-25T00:29:43.340552+0000","last_peered":"2024-04-25T00:29:43.340552+0000","last_clean":"2024-04-25T00:29:43.340552+0000","last_became_active":"2024-04-25T00:28:24.188329+0000","last_became_peered":"2024-04-25T00:28:24.188329+0000","last_unstale":"2024-04-25T00:29:43.340552+0000","last_undegraded":"2024-04-25T00:29:43.340552+0000","last_fullsized":"2024-04-25T00:29:43.340552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:41.523820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.434851+0000","last_change":"2024-04-25T00:28:23.405933+0000","last_active":"2024-04-25T00:29:14.434851+0000","last_peered":"2024-04-25T00:29:14.434851+0000","last_clean":"2024-04-25T00:29:14.434851+0000","last_became_active":"2024-04-25T00:28:23.405585+0000","last_became_peered":"2024-04-25T00:28:23.405585+0000","last_unstale":"2024-04-25T00:29:14.434851+0000","last_undegraded":"2024-04-25T00:29:14.434851+0000","last_fullsized":"2024-04-25T00:29:14.434851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:05.914735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.341372+0000","last_change":"2024-04-25T00:28:24.178720+0000","last_active":"2024-04-25T00:29:43.341372+0000","last_peered":"2024-04-25T00:29:43.341372+0000","last_clean":"2024-04-25T00:29:43.341372+0000","last_became_active":"2024-04-25T00:28:24.178270+0000","last_became_peered":"2024-04-25T00:28:24.178270+0000","last_unstale":"2024-04-25T00:29:43.341372+0000","last_undegraded":"2024-04-25T00:29:43.341372+0000","last_fullsized":"2024-04-25T00:29:43.341372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:38.076554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.433230+0000","last_change":"2024-04-25T00:28:23.403017+0000","last_active":"2024-04-25T00:29:14.433230+0000","last_peered":"2024-04-25T00:29:14.433230+0000","last_clean":"2024-04-25T00:29:14.433230+0000","last_became_active":"2024-04-25T00:28:23.402706+0000","last_became_peered":"2024-04-25T00:28:23.402706+0000","last_unstale":"2024-04-25T00:29:14.433230+0000","last_undegraded":"2024-04-25T00:29:14.433230+0000","last_fullsized":"2024-04-25T00:29:14.433230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:22.372941+0000","last_clean_scrub_stamp":"2024-04-25T00:28:22.372941+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:06.916509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.335019+0000","last_change":"2024-04-25T00:28:24.181585+0000","last_active":"2024-04-25T00:29:43.335019+0000","last_peered":"2024-04-25T00:29:43.335019+0000","last_clean":"2024-04-25T00:29:43.335019+0000","last_became_active":"2024-04-25T00:28:24.180755+0000","last_became_peered":"2024-04-25T00:28:24.180755+0000","last_unstale":"2024-04-25T00:29:43.335019+0000","last_undegraded":"2024-04-25T00:29:43.335019+0000","last_fullsized":"2024-04-25T00:29:43.335019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:59.711999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.336608+0000","last_change":"2024-04-25T00:28:24.175444+0000","last_active":"2024-04-25T00:29:43.336608+0000","last_peered":"2024-04-25T00:29:43.336608+0000","last_clean":"2024-04-25T00:29:43.336608+0000","last_became_active":"2024-04-25T00:28:24.174322+0000","last_became_peered":"2024-04-25T00:28:24.174322+0000","last_unstale":"2024-04-25T00:29:43.336608+0000","last_undegraded":"2024-04-25T00:29:43.336608+0000","last_fullsized":"2024-04-25T00:29:43.336608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:15.325122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.336863+0000","last_change":"2024-04-25T00:28:24.188603+0000","last_active":"2024-04-25T00:29:43.336863+0000","last_peered":"2024-04-25T00:29:43.336863+0000","last_clean":"2024-04-25T00:29:43.336863+0000","last_became_active":"2024-04-25T00:28:24.188300+0000","last_became_peered":"2024-04-25T00:28:24.188300+0000","last_unstale":"2024-04-25T00:29:43.336863+0000","last_undegraded":"2024-04-25T00:29:43.336863+0000","last_fullsized":"2024-04-25T00:29:43.336863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:45.466914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.336254+0000","last_change":"2024-04-25T00:28:24.183907+0000","last_active":"2024-04-25T00:29:43.336254+0000","last_peered":"2024-04-25T00:29:43.336254+0000","last_clean":"2024-04-25T00:29:43.336254+0000","last_became_active":"2024-04-25T00:28:24.183545+0000","last_became_peered":"2024-04-25T00:28:24.183545+0000","last_unstale":"2024-04-25T00:29:43.336254+0000","last_undegraded":"2024-04-25T00:29:43.336254+0000","last_fullsized":"2024-04-25T00:29:43.336254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:02.883338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.435288+0000","last_change":"2024-04-25T00:28:24.178516+0000","last_active":"2024-04-25T00:29:14.435288+0000","last_peered":"2024-04-25T00:29:14.435288+0000","last_clean":"2024-04-25T00:29:14.435288+0000","last_became_active":"2024-04-25T00:28:24.177995+0000","last_became_peered":"2024-04-25T00:28:24.177995+0000","last_unstale":"2024-04-25T00:29:14.435288+0000","last_undegraded":"2024-04-25T00:29:14.435288+0000","last_fullsized":"2024-04-25T00:29:14.435288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:03.396880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.337832+0000","last_change":"2024-04-25T00:28:24.175401+0000","last_active":"2024-04-25T00:29:43.337832+0000","last_peered":"2024-04-25T00:29:43.337832+0000","last_clean":"2024-04-25T00:29:43.337832+0000","last_became_active":"2024-04-25T00:28:24.174648+0000","last_became_peered":"2024-04-25T00:28:24.174648+0000","last_unstale":"2024-04-25T00:29:43.337832+0000","last_undegraded":"2024-04-25T00:29:43.337832+0000","last_fullsized":"2024-04-25T00:29:43.337832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:00.630810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.334751+0000","last_change":"2024-04-25T00:28:24.173542+0000","last_active":"2024-04-25T00:29:43.334751+0000","last_peered":"2024-04-25T00:29:43.334751+0000","last_clean":"2024-04-25T00:29:43.334751+0000","last_became_active":"2024-04-25T00:28:24.173240+0000","last_became_peered":"2024-04-25T00:28:24.173240+0000","last_unstale":"2024-04-25T00:29:43.334751+0000","last_undegraded":"2024-04-25T00:29:43.334751+0000","last_fullsized":"2024-04-25T00:29:43.334751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:00.189612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.334576+0000","last_change":"2024-04-25T00:28:24.185104+0000","last_active":"2024-04-25T00:29:43.334576+0000","last_peered":"2024-04-25T00:29:43.334576+0000","last_clean":"2024-04-25T00:29:43.334576+0000","last_became_active":"2024-04-25T00:28:24.184566+0000","last_became_peered":"2024-04-25T00:28:24.184566+0000","last_unstale":"2024-04-25T00:29:43.334576+0000","last_undegraded":"2024-04-25T00:29:43.334576+0000","last_fullsized":"2024-04-25T00:29:43.334576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:48.469722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.312796+0000","last_change":"2024-04-25T00:28:24.181286+0000","last_active":"2024-04-25T00:29:43.312796+0000","last_peered":"2024-04-25T00:29:43.312796+0000","last_clean":"2024-04-25T00:29:43.312796+0000","last_became_active":"2024-04-25T00:28:24.180406+0000","last_became_peered":"2024-04-25T00:28:24.180406+0000","last_unstale":"2024-04-25T00:29:43.312796+0000","last_undegraded":"2024-04-25T00:29:43.312796+0000","last_fullsized":"2024-04-25T00:29:43.312796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.644228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.441147+0000","last_change":"2024-04-25T00:29:14.440689+0000","last_active":"2024-04-25T00:29:14.441147+0000","last_peered":"2024-04-25T00:29:14.441147+0000","last_clean":"2024-04-25T00:29:14.441147+0000","last_became_active":"2024-04-25T00:29:14.440255+0000","last_became_peered":"2024-04-25T00:29:14.440255+0000","last_unstale":"2024-04-25T00:29:14.441147+0000","last_undegraded":"2024-04-25T00:29:14.441147+0000","last_fullsized":"2024-04-25T00:29:14.441147+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:37.142787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.316928+0000","last_change":"2024-04-25T00:28:24.180521+0000","last_active":"2024-04-25T00:29:43.316928+0000","last_peered":"2024-04-25T00:29:43.316928+0000","last_clean":"2024-04-25T00:29:43.316928+0000","last_became_active":"2024-04-25T00:28:24.179867+0000","last_became_peered":"2024-04-25T00:28:24.179867+0000","last_unstale":"2024-04-25T00:29:43.316928+0000","last_undegraded":"2024-04-25T00:29:43.316928+0000","last_fullsized":"2024-04-25T00:29:43.316928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:31.660585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:14.664698+0000","last_change":"2024-04-25T00:28:24.185050+0000","last_active":"2024-04-25T00:29:14.664698+0000","last_peered":"2024-04-25T00:29:14.664698+0000","last_clean":"2024-04-25T00:29:14.664698+0000","last_became_active":"2024-04-25T00:28:24.184663+0000","last_became_peered":"2024-04-25T00:28:24.184663+0000","last_unstale":"2024-04-25T00:29:14.664698+0000","last_undegraded":"2024-04-25T00:29:14.664698+0000","last_fullsized":"2024-04-25T00:29:14.664698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:32.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.338677+0000","last_change":"2024-04-25T00:28:24.177948+0000","last_active":"2024-04-25T00:29:43.338677+0000","last_peered":"2024-04-25T00:29:43.338677+0000","last_clean":"2024-04-25T00:29:43.338677+0000","last_became_active":"2024-04-25T00:28:24.177627+0000","last_became_peered":"2024-04-25T00:28:24.177627+0000","last_unstale":"2024-04-25T00:29:43.338677+0000","last_undegraded":"2024-04-25T00:29:43.338677+0000","last_fullsized":"2024-04-25T00:29:43.338677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:29.128868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:29:43.331857+0000","last_change":"2024-04-25T00:28:24.176115+0000","last_active":"2024-04-25T00:29:43.331857+0000","last_peered":"2024-04-25T00:29:43.331857+0000","last_clean":"2024-04-25T00:29:43.331857+0000","last_became_active":"2024-04-25T00:28:24.175735+0000","last_became_peered":"2024-04-25T00:28:24.175735+0000","last_unstale":"2024-04-25T00:29:43.331857+0000","last_undegraded":"2024-04-25T00:29:43.331857+0000","last_fullsized":"2024-04-25T00:29:43.331857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:28:23.381255+0000","last_clean_scrub_stamp":"2024-04-25T00:28:23.381255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:58.595542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280907,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66977,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":793,"num_read_kb":465,"num_write":238,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318300,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":465335,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1116387,"data_compressed":41565,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":465265,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005101,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":465643,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1058302,"data_compressed":41565,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":464982,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":525918,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":466321,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":526807,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":464807,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":464955,"data_compressed":5837,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1057980,"data_compressed":41565,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35728,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:29:48.291 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T00:29:48.291 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:29:48.291 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:29:48.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T00:29:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:48.921 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:48 smithi031 ceph-mon[29167]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T00:29:48.922 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:48 smithi031 ceph-mon[29167]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:29:49.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:48 smithi169 ceph-mon[33112]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T00:29:49.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:48 smithi169 ceph-mon[33112]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:29:49.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:48 smithi040 ceph-mon[32945]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T00:29:49.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:48 smithi040 ceph-mon[32945]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 -- 172.21.15.31:0/3038419138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 msgr2=0x7fc5801010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 --2- 172.21.15.31:0/3038419138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc5801010e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc574009990 tx=0x7fc574009ca0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 -- 172.21.15.31:0/3038419138 shutdown_connections 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 --2- 172.21.15.31:0/3038419138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc580101620 0x7fc58010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 --2- 172.21.15.31:0/3038419138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc580108410 0x7fc5801087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 --2- 172.21.15.31:0/3038419138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc5801010e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.249+0000 7fc586109700 1 -- 172.21.15.31:0/3038419138 >> 172.21.15.31:0/3038419138 conn(0x7fc5800763a0 msgr2=0x7fc5800767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.250+0000 7fc586109700 1 -- 172.21.15.31:0/3038419138 shutdown_connections 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.250+0000 7fc586109700 1 -- 172.21.15.31:0/3038419138 wait complete. 2024-04-25T00:29:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.250+0000 7fc586109700 1 Processor -- start 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.250+0000 7fc586109700 1 -- start start 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc58019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc580101620 0x7fc58019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc580108410 0x7fc5801986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801123c0 con 0x7fc580100c60 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc580112240 con 0x7fc580101620 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc580112540 con 0x7fc580108410 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc58019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc58019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55562/0 (socket says 172.21.15.31:55562) 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 -- 172.21.15.31:0/950922675 learned_addr learned my addr 172.21.15.31:0/950922675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57effd700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc580101620 0x7fc58019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57ffff700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc580108410 0x7fc5801986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 -- 172.21.15.31:0/950922675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc580108410 msgr2=0x7fc5801986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc580108410 0x7fc5801986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 -- 172.21.15.31:0/950922675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc580101620 msgr2=0x7fc58019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc580101620 0x7fc58019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 -- 172.21.15.31:0/950922675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580198f80 con 0x7fc580100c60 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57f7fe700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc58019e5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc57000d950 tx=0x7fc57000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570016070 con 0x7fc580100c60 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc570009df0 con 0x7fc580100c60 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc580199270 con 0x7fc580100c60 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57000f3c0 con 0x7fc580100c60 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.251+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5801a5550 con 0x7fc580100c60 2024-04-25T00:29:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.252+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58006a400 con 0x7fc580100c60 2024-04-25T00:29:50.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.254+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc570019070 con 0x7fc580100c60 2024-04-25T00:29:50.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.254+0000 7fc57cff9700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc5680a2c20 0x7fc5680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:50.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.255+0000 7fc57effd700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc5680a2c20 0x7fc5680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:50.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.255+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fc5700cec10 con 0x7fc580100c60 2024-04-25T00:29:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.255+0000 7fc57effd700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc5680a2c20 0x7fc5680a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b5c0 tx=0x7fc574005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.258+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc57009bb60 con 0x7fc580100c60 2024-04-25T00:29:50.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.512+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc580199bd0 con 0x7fc580100c60 2024-04-25T00:29:50.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.513+0000 7fc57cff9700 1 -- 172.21.15.31:0/950922675 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc57009b460 con 0x7fc580100c60 2024-04-25T00:29:50.515 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:50.515 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc5680a2c20 msgr2=0x7fc5680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc5680a2c20 0x7fc5680a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b5c0 tx=0x7fc574005960 comp rx=0 tx=0).stop 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 msgr2=0x7fc58019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc58019e5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc57000d950 tx=0x7fc57000dc60 comp rx=0 tx=0).stop 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 shutdown_connections 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc5680a2c20 0x7fc5680a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc580100c60 0x7fc58019e5d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc580108410 0x7fc5801986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 --2- 172.21.15.31:0/950922675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc580101620 0x7fc58019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 >> 172.21.15.31:0/950922675 conn(0x7fc5800763a0 msgr2=0x7fc5800ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 shutdown_connections 2024-04-25T00:29:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:50.515+0000 7fc586109700 1 -- 172.21.15.31:0/950922675 wait complete. 2024-04-25T00:29:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:50 smithi169 ceph-mon[33112]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T00:29:51.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:50 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/950922675' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:29:51.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:50 smithi040 ceph-mon[32945]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T00:29:51.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:50 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/950922675' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:29:51.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:50 smithi031 ceph-mon[29167]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T00:29:51.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:50 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/950922675' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:29:51.243 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:29:51.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:29:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.348+0000 7ffb7a109640 1 -- 172.21.15.31:0/2443145760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 msgr2=0x7ffb74101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.348+0000 7ffb7a109640 1 --2- 172.21.15.31:0/2443145760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74101bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ca0 tx=0x7ffb6402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- 172.21.15.31:0/2443145760 shutdown_connections 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 --2- 172.21.15.31:0/2443145760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb74103360 0x7ffb74109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 --2- 172.21.15.31:0/2443145760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb741029a0 0x7ffb74102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 --2- 172.21.15.31:0/2443145760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74101bd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- 172.21.15.31:0/2443145760 >> 172.21.15.31:0/2443145760 conn(0x7ffb740fcd40 msgr2=0x7ffb740ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- 172.21.15.31:0/2443145760 shutdown_connections 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- 172.21.15.31:0/2443145760 wait complete. 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 Processor -- start 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- start start 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb741029a0 0x7ffb74197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb74103360 0x7ffb7419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7410d330 con 0x7ffb741017b0 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7410d1b0 con 0x7ffb74103360 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.349+0000 7ffb7a109640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7410d4b0 con 0x7ffb741029a0 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55586/0 (socket says 172.21.15.31:55586) 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 -- 172.21.15.31:0/1380872105 learned_addr learned my addr 172.21.15.31:0/1380872105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb72ffd640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb741029a0 0x7ffb74197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb73fff640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb74103360 0x7ffb7419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 -- 172.21.15.31:0/1380872105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb741029a0 msgr2=0x7ffb74197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb741029a0 0x7ffb74197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 -- 172.21.15.31:0/1380872105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb74103360 msgr2=0x7ffb7419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb74103360 0x7ffb7419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 -- 172.21.15.31:0/1380872105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7419f620 con 0x7ffb741017b0 2024-04-25T00:29:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb737fe640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74197a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb6402aeb0 tx=0x7ffb64003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64004240 con 0x7ffb741017b0 2024-04-25T00:29:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb64032ec0 con 0x7ffb741017b0 2024-04-25T00:29:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7419f8b0 con 0x7ffb741017b0 2024-04-25T00:29:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64030db0 con 0x7ffb741017b0 2024-04-25T00:29:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.350+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7419fe50 con 0x7ffb741017b0 2024-04-25T00:29:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.351+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb74108af0 con 0x7ffb741017b0 2024-04-25T00:29:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.352+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb640305c0 con 0x7ffb741017b0 2024-04-25T00:29:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.353+0000 7ffb70ff9640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ffb5c0a2c10 0x7ffb5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.353+0000 7ffb72ffd640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ffb5c0a2c10 0x7ffb5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.353+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7ffb640f0480 con 0x7ffb741017b0 2024-04-25T00:29:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.353+0000 7ffb72ffd640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ffb5c0a2c10 0x7ffb5c0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb74198f10 tx=0x7ffb6800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.357+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb640bd4f0 con 0x7ffb741017b0 2024-04-25T00:29:51.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.543+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb74062340 con 0x7ffb741017b0 2024-04-25T00:29:51.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.546+0000 7ffb70ff9640 1 -- 172.21.15.31:0/1380872105 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22650 (secure 0 0 0) 0x7ffb640bcdf0 con 0x7ffb741017b0 2024-04-25T00:29:51.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:51.547 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:29:14.426545+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ffb5c0a2c10 msgr2=0x7ffb5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ffb5c0a2c10 0x7ffb5c0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb74198f10 tx=0x7ffb6800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 msgr2=0x7ffb74197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74197a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb6402aeb0 tx=0x7ffb64003c20 comp rx=0 tx=0).stop 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 shutdown_connections 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb74103360 0x7ffb7419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb741029a0 0x7ffb74197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ffb5c0a2c10 0x7ffb5c0a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 --2- 172.21.15.31:0/1380872105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb741017b0 0x7ffb74197a00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 >> 172.21.15.31:0/1380872105 conn(0x7ffb740fcd40 msgr2=0x7ffb74105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 shutdown_connections 2024-04-25T00:29:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.548+0000 7ffb7a109640 1 -- 172.21.15.31:0/1380872105 wait complete. 2024-04-25T00:29:51.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:29:51.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:29:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- 172.21.15.31:0/2535869614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 msgr2=0x7f8954109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- 172.21.15.31:0/2535869614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f8954109be0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8944009ca0 tx=0x7f894402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- 172.21.15.31:0/2535869614 shutdown_connections 2024-04-25T00:29:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- 172.21.15.31:0/2535869614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f8954109be0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- 172.21.15.31:0/2535869614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8954102990 0x7f8954102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- 172.21.15.31:0/2535869614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89541017a0 0x7f8954101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- 172.21.15.31:0/2535869614 >> 172.21.15.31:0/2535869614 conn(0x7f89540fcd10 msgr2=0x7f89540ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- 172.21.15.31:0/2535869614 shutdown_connections 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- 172.21.15.31:0/2535869614 wait complete. 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 Processor -- start 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- start start 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89541017a0 0x7f8954197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8954102990 0x7f8954197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f895419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f895410d320 con 0x7f8954103350 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f895410d1a0 con 0x7f89541017a0 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.664+0000 7f895ba31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f895410d4a0 con 0x7f8954102990 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f895419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f895419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55612/0 (socket says 172.21.15.31:55612) 2024-04-25T00:29:51.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 -- 172.21.15.31:0/2619616301 learned_addr learned my addr 172.21.15.31:0/2619616301 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f89597cd640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89541017a0 0x7f8954197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8958fcc640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8954102990 0x7f8954197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 -- 172.21.15.31:0/2619616301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8954102990 msgr2=0x7f8954197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8954102990 0x7f8954197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 -- 172.21.15.31:0/2619616301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89541017a0 msgr2=0x7f8954197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89541017a0 0x7f8954197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 -- 172.21.15.31:0/2619616301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895419f520 con 0x7f8954103350 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f89597cd640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89541017a0 0x7f8954197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f8959fce640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f895419eeb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8944031040 tx=0x7f8944003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944032ec0 con 0x7f8954103350 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89440044f0 con 0x7f8954103350 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895419f7b0 con 0x7f8954103350 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944030d10 con 0x7f8954103350 2024-04-25T00:29:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.665+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895419fd50 con 0x7f8954103350 2024-04-25T00:29:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.666+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8954108a60 con 0x7f8954103350 2024-04-25T00:29:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.667+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8944004040 con 0x7f8954103350 2024-04-25T00:29:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.668+0000 7f89527fc640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f893c0a2db0 0x7f893c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.668+0000 7f89597cd640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f893c0a2db0 0x7f893c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.668+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f89440f08e0 con 0x7f8954103350 2024-04-25T00:29:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.669+0000 7f89597cd640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f893c0a2db0 0x7f893c0a5270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f89541027f0 tx=0x7f894c005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.673+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f89440bd7d0 con 0x7f8954103350 2024-04-25T00:29:51.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.861+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8954062340 con 0x7f8954103350 2024-04-25T00:29:51.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.861+0000 7f89527fc640 1 -- 172.21.15.31:0/2619616301 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f89440bd0d0 con 0x7f8954103350 2024-04-25T00:29:51.863 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:29:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.863+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f893c0a2db0 msgr2=0x7f893c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.863+0000 7f895ba31640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f893c0a2db0 0x7f893c0a5270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f89541027f0 tx=0x7f894c005e70 comp rx=0 tx=0).stop 2024-04-25T00:29:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.863+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 msgr2=0x7f895419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.863+0000 7f895ba31640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f895419eeb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8944031040 tx=0x7f8944003ba0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 shutdown_connections 2024-04-25T00:29:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f893c0a2db0 0x7f893c0a5270 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8954103350 0x7f895419eeb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8954102990 0x7f8954197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 --2- 172.21.15.31:0/2619616301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89541017a0 0x7f8954197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 >> 172.21.15.31:0/2619616301 conn(0x7f89540fcd10 msgr2=0x7f8954105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 shutdown_connections 2024-04-25T00:29:51.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.864+0000 7f895ba31640 1 -- 172.21.15.31:0/2619616301 wait complete. 2024-04-25T00:29:51.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 -- 172.21.15.31:0/3744810201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c1017a0 msgr2=0x7f744c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 --2- 172.21.15.31:0/3744810201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c1017a0 0x7f744c101bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7448009900 tx=0x7f744802a410 comp rx=0 tx=0).stop 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 -- 172.21.15.31:0/3744810201 shutdown_connections 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 --2- 172.21.15.31:0/3744810201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f744c103350 0x7f744c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 --2- 172.21.15.31:0/3744810201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c102990 0x7f744c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 --2- 172.21.15.31:0/3744810201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c1017a0 0x7f744c101bc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 -- 172.21.15.31:0/3744810201 >> 172.21.15.31:0/3744810201 conn(0x7f744c0fcd10 msgr2=0x7f744c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 -- 172.21.15.31:0/3744810201 shutdown_connections 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.980+0000 7f74544fc640 1 -- 172.21.15.31:0/3744810201 wait complete. 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 Processor -- start 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 -- start start 2024-04-25T00:29:51.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c1017a0 0x7f744c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f744c102990 0x7f744c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 0x7f744c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f744c10d2c0 con 0x7f744c103350 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f744c10d140 con 0x7f744c102990 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74544fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f744c10d440 con 0x7f744c1017a0 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 0x7f744c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 0x7f744c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55634/0 (socket says 172.21.15.31:55634) 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 -- 172.21.15.31:0/2617860260 learned_addr learned my addr 172.21.15.31:0/2617860260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452298640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c1017a0 0x7f744c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7451a97640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f744c102990 0x7f744c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 -- 172.21.15.31:0/2617860260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c1017a0 msgr2=0x7f744c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c1017a0 0x7f744c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 -- 172.21.15.31:0/2617860260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f744c102990 msgr2=0x7f744c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f744c102990 0x7f744c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 -- 172.21.15.31:0/2617860260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c19f670 con 0x7f744c103350 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452298640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c1017a0 0x7f744c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f7452a99640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 0x7f744c19f000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7444009fd0 tx=0x7f744400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.981+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444017070 con 0x7f744c103350 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.982+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7444003e20 con 0x7f744c103350 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.982+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c19f960 con 0x7f744c103350 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.982+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444010460 con 0x7f744c103350 2024-04-25T00:29:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.982+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c19fe60 con 0x7f744c103350 2024-04-25T00:29:51.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.983+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744c108c30 con 0x7f744c103350 2024-04-25T00:29:51.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.984+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7444003970 con 0x7f744c103350 2024-04-25T00:29:51.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.984+0000 7f74437fe640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74380a2b90 0x7f74380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.985+0000 7f7452298640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74380a2b90 0x7f74380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.985+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f74440cef50 con 0x7f744c103350 2024-04-25T00:29:51.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.985+0000 7f7452298640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74380a2b90 0x7f74380a5050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7448000c00 tx=0x7f7448032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:51.989+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f744409bea0 con 0x7f744c103350 2024-04-25T00:29:52.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:51 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1380872105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:52.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:51 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1380872105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:52.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.173+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f744c062340 con 0x7f744c103350 2024-04-25T00:29:52.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.173+0000 7f74437fe640 1 -- 172.21.15.31:0/2617860260 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f744409b7a0 con 0x7f744c103350 2024-04-25T00:29:52.174 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:29:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74380a2b90 msgr2=0x7f74380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74380a2b90 0x7f74380a5050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7448000c00 tx=0x7f7448032c40 comp rx=0 tx=0).stop 2024-04-25T00:29:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 msgr2=0x7f744c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 0x7f744c19f000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7444009fd0 tx=0x7f744400b260 comp rx=0 tx=0).stop 2024-04-25T00:29:52.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 shutdown_connections 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f74380a2b90 0x7f74380a5050 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f744c103350 0x7f744c19f000 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f744c102990 0x7f744c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 --2- 172.21.15.31:0/2617860260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c1017a0 0x7f744c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 >> 172.21.15.31:0/2617860260 conn(0x7f744c0fcd10 msgr2=0x7f744c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 shutdown_connections 2024-04-25T00:29:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.176+0000 7f74544fc640 1 -- 172.21.15.31:0/2617860260 wait complete. 2024-04-25T00:29:52.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:51 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1380872105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:52.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 -- 172.21.15.31:0/3682230941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 msgr2=0x7fd54c109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 --2- 172.21.15.31:0/3682230941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c109790 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd544009ca0 tx=0x7fd54402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 -- 172.21.15.31:0/3682230941 shutdown_connections 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 --2- 172.21.15.31:0/3682230941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c109790 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 --2- 172.21.15.31:0/3682230941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd54c104a10 0x7fd54c106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 --2- 172.21.15.31:0/3682230941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c1020e0 0x7fd54c1044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 -- 172.21.15.31:0/3682230941 >> 172.21.15.31:0/3682230941 conn(0x7fd54c0fbaf0 msgr2=0x7fd54c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 -- 172.21.15.31:0/3682230941 shutdown_connections 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.293+0000 7fd55219e640 1 -- 172.21.15.31:0/3682230941 wait complete. 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 Processor -- start 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 -- start start 2024-04-25T00:29:52.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c1020e0 0x7fd54c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd54c104a10 0x7fd54c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d1a0 con 0x7fd54c107340 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d020 con 0x7fd54c1020e0 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55219e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d320 con 0x7fd54c104a10 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55664/0 (socket says 172.21.15.31:55664) 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 -- 172.21.15.31:0/652855257 learned_addr learned my addr 172.21.15.31:0/652855257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55099b640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd54c104a10 0x7fd54c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55119c640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c1020e0 0x7fd54c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 -- 172.21.15.31:0/652855257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd54c104a10 msgr2=0x7fd54c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd54c104a10 0x7fd54c197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 -- 172.21.15.31:0/652855257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c1020e0 msgr2=0x7fd54c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c1020e0 0x7fd54c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.294+0000 7fd55199d640 1 -- 172.21.15.31:0/652855257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c19f440 con 0x7fd54c107340 2024-04-25T00:29:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.295+0000 7fd55199d640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c19edd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd544009dc0 tx=0x7fd544003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.295+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544032ec0 con 0x7fd54c107340 2024-04-25T00:29:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.295+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5440044f0 con 0x7fd54c107340 2024-04-25T00:29:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.295+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c19f6d0 con 0x7fd54c107340 2024-04-25T00:29:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.295+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544030d40 con 0x7fd54c107340 2024-04-25T00:29:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.295+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c19fc20 con 0x7fd54c107340 2024-04-25T00:29:52.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.296+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c0fd5e0 con 0x7fd54c107340 2024-04-25T00:29:52.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.297+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd544004040 con 0x7fd54c107340 2024-04-25T00:29:52.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.298+0000 7fd5427fc640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd5380a2d30 0x7fd5380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.298+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fd5440f0670 con 0x7fd54c107340 2024-04-25T00:29:52.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.298+0000 7fd55119c640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd5380a2d30 0x7fd5380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.298+0000 7fd55119c640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd5380a2d30 0x7fd5380a51f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd548005d00 tx=0x7fd548005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:52.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.302+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd5440bd5c0 con 0x7fd54c107340 2024-04-25T00:29:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.486+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd54c062340 con 0x7fd54c107340 2024-04-25T00:29:52.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.487+0000 7fd5427fc640 1 -- 172.21.15.31:0/652855257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fd5440bcec0 con 0x7fd54c107340 2024-04-25T00:29:52.487 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:29:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd5380a2d30 msgr2=0x7fd5380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd5380a2d30 0x7fd5380a51f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd548005d00 tx=0x7fd548005c50 comp rx=0 tx=0).stop 2024-04-25T00:29:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 msgr2=0x7fd54c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c19edd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd544009dc0 tx=0x7fd544003ba0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 shutdown_connections 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd5380a2d30 0x7fd5380a51f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd54c107340 0x7fd54c19edd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd54c104a10 0x7fd54c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 --2- 172.21.15.31:0/652855257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c1020e0 0x7fd54c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 >> 172.21.15.31:0/652855257 conn(0x7fd54c0fbaf0 msgr2=0x7fd54c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 shutdown_connections 2024-04-25T00:29:52.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.489+0000 7fd55219e640 1 -- 172.21.15.31:0/652855257 wait complete. 2024-04-25T00:29:52.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 -- 172.21.15.31:0/1824023502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 msgr2=0x7f7378108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 --2- 172.21.15.31:0/1824023502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f7378108ae0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7370009ca0 tx=0x7f737002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 -- 172.21.15.31:0/1824023502 shutdown_connections 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 --2- 172.21.15.31:0/1824023502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f7378108ae0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 --2- 172.21.15.31:0/1824023502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7378101890 0x7f7378101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 --2- 172.21.15.31:0/1824023502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73781006a0 0x7f7378100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 -- 172.21.15.31:0/1824023502 >> 172.21.15.31:0/1824023502 conn(0x7f73780fbc30 msgr2=0x7f73780fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 -- 172.21.15.31:0/1824023502 shutdown_connections 2024-04-25T00:29:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.601+0000 7f737e2b3640 1 -- 172.21.15.31:0/1824023502 wait complete. 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 Processor -- start 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 -- start start 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73781006a0 0x7f7378197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7378101890 0x7f7378197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f737819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f737810d350 con 0x7f7378102250 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f737810d1d0 con 0x7f7378101890 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737e2b3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f737810d4d0 con 0x7f73781006a0 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737c850640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f737819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737c850640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f737819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55690/0 (socket says 172.21.15.31:55690) 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f737c850640 1 -- 172.21.15.31:0/829768143 learned_addr learned my addr 172.21.15.31:0/829768143 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.602+0000 7f73777fe640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7378101890 0x7f7378197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f7377fff640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73781006a0 0x7f7378197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737c850640 1 -- 172.21.15.31:0/829768143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73781006a0 msgr2=0x7f7378197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737c850640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73781006a0 0x7f7378197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737c850640 1 -- 172.21.15.31:0/829768143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7378101890 msgr2=0x7f7378197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737c850640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7378101890 0x7f7378197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737c850640 1 -- 172.21.15.31:0/829768143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737819f5a0 con 0x7f7378102250 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737c850640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f737819ef30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7370031040 tx=0x7f7370003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370032ec0 con 0x7f7378102250 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73700044f0 con 0x7f7378102250 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737819f830 con 0x7f7378102250 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370030cb0 con 0x7f7378102250 2024-04-25T00:29:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.603+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737819fdd0 con 0x7f7378102250 2024-04-25T00:29:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.604+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73781079c0 con 0x7f7378102250 2024-04-25T00:29:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.605+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7370004040 con 0x7f7378102250 2024-04-25T00:29:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.606+0000 7f73757fa640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73640a2db0 0x7f73640a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.606+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f73700f05e0 con 0x7f7378102250 2024-04-25T00:29:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.606+0000 7f7377fff640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73640a2db0 0x7f73640a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.606+0000 7f7377fff640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73640a2db0 0x7f73640a5270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73781016f0 tx=0x7f7360005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.610+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73700bd4d0 con 0x7f7378102250 2024-04-25T00:29:52.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.833+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7378062340 con 0x7f7378102250 2024-04-25T00:29:52.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.835+0000 7f73757fa640 1 -- 172.21.15.31:0/829768143 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11926 (secure 0 0 0) 0x7f73700bcdd0 con 0x7f7378102250 2024-04-25T00:29:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:52.837 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:28:30.181657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24512,24517,24499,24434,24424]},"id":1}]} 2024-04-25T00:29:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:29:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.837+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73640a2db0 msgr2=0x7f73640a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.837+0000 7f737e2b3640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73640a2db0 0x7f73640a5270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73781016f0 tx=0x7f7360005f90 comp rx=0 tx=0).stop 2024-04-25T00:29:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.837+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 msgr2=0x7f737819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.837+0000 7f737e2b3640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f737819ef30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7370031040 tx=0x7f7370003ba0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 shutdown_connections 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f73640a2db0 0x7f73640a5270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378102250 0x7f737819ef30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7378101890 0x7f7378197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 --2- 172.21.15.31:0/829768143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73781006a0 0x7f7378197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 >> 172.21.15.31:0/829768143 conn(0x7f73780fbc30 msgr2=0x7f7378104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 shutdown_connections 2024-04-25T00:29:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.838+0000 7f737e2b3640 1 -- 172.21.15.31:0/829768143 wait complete. 2024-04-25T00:29:52.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 -- 172.21.15.31:0/4168074017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 msgr2=0x7f6578109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 --2- 172.21.15.31:0/4168074017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578109bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6570009ca0 tx=0x7f657002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 -- 172.21.15.31:0/4168074017 shutdown_connections 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 --2- 172.21.15.31:0/4168074017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578109bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 --2- 172.21.15.31:0/4168074017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65781029a0 0x7f6578102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 --2- 172.21.15.31:0/4168074017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 0x7f6578101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 -- 172.21.15.31:0/4168074017 >> 172.21.15.31:0/4168074017 conn(0x7f65780fcd40 msgr2=0x7f65780ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 -- 172.21.15.31:0/4168074017 shutdown_connections 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.951+0000 7f657f363640 1 -- 172.21.15.31:0/4168074017 wait complete. 2024-04-25T00:29:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 Processor -- start 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 -- start start 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 0x7f65780754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65781029a0 0x7f6578073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f657810d2e0 con 0x7f6578103360 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f657810d160 con 0x7f65781017b0 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657f363640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f657810d460 con 0x7f65781029a0 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47610/0 (socket says 172.21.15.31:47610) 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 -- 172.21.15.31:0/3131774726 learned_addr learned my addr 172.21.15.31:0/3131774726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657c8fe640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65781029a0 0x7f6578073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d0ff640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 0x7f65780754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 -- 172.21.15.31:0/3131774726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65781029a0 msgr2=0x7f6578073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65781029a0 0x7f6578073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 -- 172.21.15.31:0/3131774726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 msgr2=0x7f65780754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 0x7f65780754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 -- 172.21.15.31:0/3131774726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6578074900 con 0x7f6578103360 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d0ff640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 0x7f65780754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.952+0000 7f657d900640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578074070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6570009dc0 tx=0x7f6570003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.953+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570032ec0 con 0x7f6578103360 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.953+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65700044f0 con 0x7f6578103360 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.953+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65781a1aa0 con 0x7f6578103360 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.953+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570030d40 con 0x7f6578103360 2024-04-25T00:29:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.953+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65781a1ff0 con 0x7f6578103360 2024-04-25T00:29:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.954+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65780689b0 con 0x7f6578103360 2024-04-25T00:29:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.955+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6570004040 con 0x7f6578103360 2024-04-25T00:29:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.956+0000 7f656e7fc640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f65640a2d30 0x7f65640a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.956+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f65700f0870 con 0x7f6578103360 2024-04-25T00:29:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.956+0000 7f657d0ff640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f65640a2d30 0x7f65640a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.956+0000 7f657d0ff640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f65640a2d30 0x7f65640a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65740058c0 tx=0x7f6574005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:52.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:52.960+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f65700bd760 con 0x7f6578103360 2024-04-25T00:29:53.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:52 smithi169 ceph-mon[33112]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:53.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:52 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2619616301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:52 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2617860260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:52 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/652855257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:52 smithi040 ceph-mon[32945]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:52 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2619616301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:52 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2617860260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:52 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/652855257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.145+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6578062340 con 0x7f6578103360 2024-04-25T00:29:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.146+0000 7f656e7fc640 1 -- 172.21.15.31:0/3131774726 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22650 (secure 0 0 0) 0x7f65700bd060 con 0x7f6578103360 2024-04-25T00:29:53.147 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:53.147 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:29:14.426545+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:29:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f65640a2d30 msgr2=0x7f65640a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f65640a2d30 0x7f65640a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65740058c0 tx=0x7f6574005850 comp rx=0 tx=0).stop 2024-04-25T00:29:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 msgr2=0x7f6578074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578074070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6570009dc0 tx=0x7f6570003ba0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 shutdown_connections 2024-04-25T00:29:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f65640a2d30 0x7f65640a51f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6578103360 0x7f6578074070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65781029a0 0x7f6578073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 --2- 172.21.15.31:0/3131774726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65781017b0 0x7f65780754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.148+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 >> 172.21.15.31:0/3131774726 conn(0x7f65780fcd40 msgr2=0x7f6578105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.149+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 shutdown_connections 2024-04-25T00:29:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.149+0000 7f657f363640 1 -- 172.21.15.31:0/3131774726 wait complete. 2024-04-25T00:29:53.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:52 smithi031 ceph-mon[29167]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:53.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:52 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2619616301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:52 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2617860260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:52 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/652855257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:29:53.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:29:53.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 -- 172.21.15.31:0/3341400712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe834101890 msgr2=0x7fe834101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 --2- 172.21.15.31:0/3341400712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe834101890 0x7fe834101d10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009900 tx=0x7fe81c02a410 comp rx=0 tx=0).stop 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 -- 172.21.15.31:0/3341400712 shutdown_connections 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 --2- 172.21.15.31:0/3341400712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe834102250 0x7fe834108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 --2- 172.21.15.31:0/3341400712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe834101890 0x7fe834101d10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 --2- 172.21.15.31:0/3341400712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8341006a0 0x7fe834100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 -- 172.21.15.31:0/3341400712 >> 172.21.15.31:0/3341400712 conn(0x7fe8340fbc10 msgr2=0x7fe8340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:53.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 -- 172.21.15.31:0/3341400712 shutdown_connections 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.267+0000 7fe838ba9640 1 -- 172.21.15.31:0/3341400712 wait complete. 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 Processor -- start 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 -- start start 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 0x7fe834104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834101890 0x7fe834102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe834102250 0x7fe8341031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe83410c270 con 0x7fe8341006a0 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe83410c0f0 con 0x7fe834101890 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe838ba9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe83410c3f0 con 0x7fe834102250 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 0x7fe834104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 0x7fe834104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47626/0 (socket says 172.21.15.31:47626) 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 -- 172.21.15.31:0/1183375463 learned_addr learned my addr 172.21.15.31:0/1183375463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe831d9b640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834101890 0x7fe834102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe832d9d640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe834102250 0x7fe8341031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 -- 172.21.15.31:0/1183375463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe834102250 msgr2=0x7fe8341031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe834102250 0x7fe8341031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 -- 172.21.15.31:0/1183375463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834101890 msgr2=0x7fe834102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.268+0000 7fe83259c640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834101890 0x7fe834102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe83259c640 1 -- 172.21.15.31:0/1183375463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe834103a40 con 0x7fe8341006a0 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe831d9b640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834101890 0x7fe834102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe83259c640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 0x7fe834104630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00b450 tx=0x7fe82c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c015020 con 0x7fe8341006a0 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8341a1be0 con 0x7fe8341006a0 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe82c011d40 con 0x7fe8341006a0 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00f340 con 0x7fe8341006a0 2024-04-25T00:29:53.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.269+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8341a2180 con 0x7fe8341006a0 2024-04-25T00:29:53.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.270+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8341a2350 con 0x7fe8341006a0 2024-04-25T00:29:53.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.271+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe82c018020 con 0x7fe8341006a0 2024-04-25T00:29:53.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.272+0000 7fe82b7fe640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe8200a2b90 0x7fe8200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.272+0000 7fe831d9b640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe8200a2b90 0x7fe8200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.272+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe82c0cebe0 con 0x7fe8341006a0 2024-04-25T00:29:53.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.272+0000 7fe831d9b640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe8200a2b90 0x7fe8200a5050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8341043a0 tx=0x7fe81c032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:53.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.277+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe82c01f020 con 0x7fe8341006a0 2024-04-25T00:29:53.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.461+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8341a2520 con 0x7fe8341006a0 2024-04-25T00:29:53.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.462+0000 7fe82b7fe640 1 -- 172.21.15.31:0/1183375463 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fe82c00d040 con 0x7fe8341006a0 2024-04-25T00:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:29:53.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe8200a2b90 msgr2=0x7fe8200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe8200a2b90 0x7fe8200a5050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8341043a0 tx=0x7fe81c032d20 comp rx=0 tx=0).stop 2024-04-25T00:29:53.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 msgr2=0x7fe834104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 0x7fe834104630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00b450 tx=0x7fe82c00b910 comp rx=0 tx=0).stop 2024-04-25T00:29:53.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 shutdown_connections 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe834102250 0x7fe8341031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834101890 0x7fe834102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe8200a2b90 0x7fe8200a5050 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 --2- 172.21.15.31:0/1183375463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8341006a0 0x7fe834104630 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 >> 172.21.15.31:0/1183375463 conn(0x7fe8340fbc10 msgr2=0x7fe8340fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 shutdown_connections 2024-04-25T00:29:53.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.464+0000 7fe838ba9640 1 -- 172.21.15.31:0/1183375463 wait complete. 2024-04-25T00:29:53.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:29:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 -- 172.21.15.31:0/3254630481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c101790 msgr2=0x7fa68c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 --2- 172.21.15.31:0/3254630481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c101790 0x7fa68c101bb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa688009900 tx=0x7fa68802a410 comp rx=0 tx=0).stop 2024-04-25T00:29:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 -- 172.21.15.31:0/3254630481 shutdown_connections 2024-04-25T00:29:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 --2- 172.21.15.31:0/3254630481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c103340 0x7fa68c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 --2- 172.21.15.31:0/3254630481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 0x7fa68c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 --2- 172.21.15.31:0/3254630481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c101790 0x7fa68c101bb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.579+0000 7fa69382a640 1 -- 172.21.15.31:0/3254630481 >> 172.21.15.31:0/3254630481 conn(0x7fa68c0fcd20 msgr2=0x7fa68c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 -- 172.21.15.31:0/3254630481 shutdown_connections 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 -- 172.21.15.31:0/3254630481 wait complete. 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 Processor -- start 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 -- start start 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c101790 0x7fa68c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 0x7fa68c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 0x7fa68c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c10d290 con 0x7fa68c103340 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c10d110 con 0x7fa68c102980 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa69382a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c10d410 con 0x7fa68c101790 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa691dc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 0x7fa68c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa691dc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 0x7fa68c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47640/0 (socket says 172.21.15.31:47640) 2024-04-25T00:29:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.580+0000 7fa691dc7640 1 -- 172.21.15.31:0/379972302 learned_addr learned my addr 172.21.15.31:0/379972302 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa690dc5640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 0x7fa68c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa6915c6640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c101790 0x7fa68c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa691dc7640 1 -- 172.21.15.31:0/379972302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c101790 msgr2=0x7fa68c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa691dc7640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c101790 0x7fa68c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa691dc7640 1 -- 172.21.15.31:0/379972302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 msgr2=0x7fa68c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa691dc7640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 0x7fa68c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa691dc7640 1 -- 172.21.15.31:0/379972302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c19f5e0 con 0x7fa68c103340 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa690dc5640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 0x7fa68c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa691dc7640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 0x7fa68c19ef70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa68400cad0 tx=0x7fa68400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68400ae20 con 0x7fa68c103340 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa684005020 con 0x7fa68c103340 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68400f6b0 con 0x7fa68c103340 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68c19f8d0 con 0x7fa68c103340 2024-04-25T00:29:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.581+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68c19fdd0 con 0x7fa68c103340 2024-04-25T00:29:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.582+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68c108b20 con 0x7fa68c103340 2024-04-25T00:29:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.583+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa6840051d0 con 0x7fa68c103340 2024-04-25T00:29:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.584+0000 7fa6827fc640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa6780a2b90 0x7fa6780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.584+0000 7fa6915c6640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa6780a2b90 0x7fa6780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.584+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fa6840cf1a0 con 0x7fa68c103340 2024-04-25T00:29:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.584+0000 7fa6915c6640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa6780a2b90 0x7fa6780a5050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa688000c00 tx=0x7fa6880314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.588+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa684014030 con 0x7fa68c103340 2024-04-25T00:29:53.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.774+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa68c062340 con 0x7fa68c103340 2024-04-25T00:29:53.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.774+0000 7fa6827fc640 1 -- 172.21.15.31:0/379972302 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fa68409c0f0 con 0x7fa68c103340 2024-04-25T00:29:53.778 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:29:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa6780a2b90 msgr2=0x7fa6780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa6780a2b90 0x7fa6780a5050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa688000c00 tx=0x7fa6880314a0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 msgr2=0x7fa68c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 0x7fa68c19ef70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa68400cad0 tx=0x7fa68400cf90 comp rx=0 tx=0).stop 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 shutdown_connections 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa6780a2b90 0x7fa6780a5050 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c103340 0x7fa68c19ef70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa68c102980 0x7fa68c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 --2- 172.21.15.31:0/379972302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c101790 0x7fa68c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 >> 172.21.15.31:0/379972302 conn(0x7fa68c0fcd20 msgr2=0x7fa68c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 shutdown_connections 2024-04-25T00:29:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.776+0000 7fa69382a640 1 -- 172.21.15.31:0/379972302 wait complete. 2024-04-25T00:29:53.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 -- 172.21.15.31:0/3245163446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258100680 msgr2=0x7fd258100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3245163446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258100680 0x7fd258100aa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd250009900 tx=0x7fd25002a410 comp rx=0 tx=0).stop 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 -- 172.21.15.31:0/3245163446 shutdown_connections 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3245163446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd258102230 0x7fd258108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3245163446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd258101870 0x7fd258101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3245163446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258100680 0x7fd258100aa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 -- 172.21.15.31:0/3245163446 >> 172.21.15.31:0/3245163446 conn(0x7fd2580fbbf0 msgr2=0x7fd2580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 -- 172.21.15.31:0/3245163446 shutdown_connections 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.892+0000 7fd25cab3640 1 -- 172.21.15.31:0/3245163446 wait complete. 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 Processor -- start 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 -- start start 2024-04-25T00:29:53.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd258100680 0x7fd2581979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 0x7fd258197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd258102230 0x7fd25819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810c180 con 0x7fd258101870 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810c000 con 0x7fd258100680 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25cab3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810c300 con 0x7fd258102230 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd255d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 0x7fd258197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd255d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 0x7fd258197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47648/0 (socket says 172.21.15.31:47648) 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd255d9b640 1 -- 172.21.15.31:0/3557827172 learned_addr learned my addr 172.21.15.31:0/3557827172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd25659c640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd258100680 0x7fd2581979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.893+0000 7fd256d9d640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd258102230 0x7fd25819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd255d9b640 1 -- 172.21.15.31:0/3557827172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd258102230 msgr2=0x7fd25819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd255d9b640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd258102230 0x7fd25819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd255d9b640 1 -- 172.21.15.31:0/3557827172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd258100680 msgr2=0x7fd2581979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd255d9b640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd258100680 0x7fd2581979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd255d9b640 1 -- 172.21.15.31:0/3557827172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25819f550 con 0x7fd258101870 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd255d9b640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 0x7fd258197ee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c920 tx=0x7fd24000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240007c70 con 0x7fd258101870 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25819f840 con 0x7fd258101870 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd240007e10 con 0x7fd258101870 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240018820 con 0x7fd258101870 2024-04-25T00:29:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.894+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25819fd90 con 0x7fd258101870 2024-04-25T00:29:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.895+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25819fff0 con 0x7fd258101870 2024-04-25T00:29:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.896+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd2400189c0 con 0x7fd258101870 2024-04-25T00:29:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.897+0000 7fd24f7fe640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd2440a2b90 0x7fd2440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.897+0000 7fd25659c640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd2440a2b90 0x7fd2440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.897+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fd2400cf470 con 0x7fd258101870 2024-04-25T00:29:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.898+0000 7fd25659c640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd2440a2b90 0x7fd2440a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd250006010 tx=0x7fd250000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:53.900+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd240014030 con 0x7fd258101870 2024-04-25T00:29:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.084+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd2581a01c0 con 0x7fd258101870 2024-04-25T00:29:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.084+0000 7fd24f7fe640 1 -- 172.21.15.31:0/3557827172 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fd24009c450 con 0x7fd258101870 2024-04-25T00:29:54.085 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd2440a2b90 msgr2=0x7fd2440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd2440a2b90 0x7fd2440a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd250006010 tx=0x7fd250000bc0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 msgr2=0x7fd258197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 0x7fd258197ee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c920 tx=0x7fd24000cde0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 shutdown_connections 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd258102230 0x7fd25819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd2440a2b90 0x7fd2440a5050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd258101870 0x7fd258197ee0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 --2- 172.21.15.31:0/3557827172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd258100680 0x7fd2581979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 >> 172.21.15.31:0/3557827172 conn(0x7fd2580fbbf0 msgr2=0x7fd2580fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 shutdown_connections 2024-04-25T00:29:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.086+0000 7fd25cab3640 1 -- 172.21.15.31:0/3557827172 wait complete. 2024-04-25T00:29:54.096 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:53 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/829768143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:54.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:53 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3131774726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:54.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:53 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1183375463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:29:54.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:53 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/379972302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:29:54.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:29:54.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:53 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/829768143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:54.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:53 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3131774726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:54.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:53 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1183375463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:29:54.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:53 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/379972302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:29:54.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:53 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/829768143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:54.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:53 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3131774726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:54.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:53 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1183375463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:29:54.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:53 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/379972302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:29:54.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.198+0000 7f8d49209640 1 -- 172.21.15.31:0/901555810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d44102250 msgr2=0x7f8d44108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.198+0000 7f8d49209640 1 --2- 172.21.15.31:0/901555810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d44102250 0x7f8d44108ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009900 tx=0x7f8d3c02a410 comp rx=0 tx=0).stop 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.198+0000 7f8d49209640 1 -- 172.21.15.31:0/901555810 shutdown_connections 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 --2- 172.21.15.31:0/901555810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d44102250 0x7f8d44108ae0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 --2- 172.21.15.31:0/901555810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d44101890 0x7f8d44101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 --2- 172.21.15.31:0/901555810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d441006a0 0x7f8d44100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 -- 172.21.15.31:0/901555810 >> 172.21.15.31:0/901555810 conn(0x7f8d440fbc30 msgr2=0x7f8d440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 -- 172.21.15.31:0/901555810 shutdown_connections 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 -- 172.21.15.31:0/901555810 wait complete. 2024-04-25T00:29:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 Processor -- start 2024-04-25T00:29:54.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.199+0000 7f8d49209640 1 -- start start 2024-04-25T00:29:54.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 0x7f8d44197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44101890 0x7f8d44197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d44102250 0x7f8d4419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4410c1d0 con 0x7f8d441006a0 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4410c050 con 0x7f8d44101890 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4410c350 con 0x7f8d44102250 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 0x7f8d44197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 0x7f8d44197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47660/0 (socket says 172.21.15.31:47660) 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 -- 172.21.15.31:0/4204288362 learned_addr learned my addr 172.21.15.31:0/4204288362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d4259c640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44101890 0x7f8d44197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d4359e640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d44102250 0x7f8d4419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 -- 172.21.15.31:0/4204288362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d44102250 msgr2=0x7f8d4419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d44102250 0x7f8d4419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 -- 172.21.15.31:0/4204288362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44101890 msgr2=0x7f8d44197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44101890 0x7f8d44197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 -- 172.21.15.31:0/4204288362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4419f5d0 con 0x7f8d441006a0 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d42d9d640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 0x7f8d44197a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400ebc0 tx=0x7f8d3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:54.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34003b70 con 0x7f8d441006a0 2024-04-25T00:29:54.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d34003d10 con 0x7f8d441006a0 2024-04-25T00:29:54.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4419f8c0 con 0x7f8d441006a0 2024-04-25T00:29:54.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.200+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34019840 con 0x7f8d441006a0 2024-04-25T00:29:54.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.201+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4419fe10 con 0x7f8d441006a0 2024-04-25T00:29:54.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.202+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d441a00a0 con 0x7f8d441006a0 2024-04-25T00:29:54.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.203+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d340199e0 con 0x7f8d441006a0 2024-04-25T00:29:54.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.203+0000 7f8d3bfff640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8d300a2b90 0x7f8d300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.203+0000 7f8d4259c640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8d300a2b90 0x7f8d300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.203+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f8d340cfaa0 con 0x7f8d441006a0 2024-04-25T00:29:54.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.204+0000 7f8d4259c640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8d300a2b90 0x7f8d300a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005fd0 tx=0x7f8d2c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.208+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8d3409c9f0 con 0x7f8d441006a0 2024-04-25T00:29:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.431+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d441a0270 con 0x7f8d441006a0 2024-04-25T00:29:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.432+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4204288362 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11926 (secure 0 0 0) 0x7f8d3409c2f0 con 0x7f8d441006a0 2024-04-25T00:29:54.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:54.433 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:28:30.181657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24512,24517,24499,24434,24424]},"id":1}]} 2024-04-25T00:29:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8d300a2b90 msgr2=0x7f8d300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8d300a2b90 0x7f8d300a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005fd0 tx=0x7f8d2c005850 comp rx=0 tx=0).stop 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 msgr2=0x7f8d44197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 0x7f8d44197a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400ebc0 tx=0x7f8d3400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 shutdown_connections 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d44102250 0x7f8d4419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44101890 0x7f8d44197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8d300a2b90 0x7f8d300a5050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 --2- 172.21.15.31:0/4204288362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d441006a0 0x7f8d44197a00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 >> 172.21.15.31:0/4204288362 conn(0x7f8d440fbc30 msgr2=0x7f8d44069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 shutdown_connections 2024-04-25T00:29:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.434+0000 7f8d49209640 1 -- 172.21.15.31:0/4204288362 wait complete. 2024-04-25T00:29:54.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 -- 172.21.15.31:0/2277301610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c1006a0 msgr2=0x7f272c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 --2- 172.21.15.31:0/2277301610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c1006a0 0x7f272c100ac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2728009900 tx=0x7f272802a410 comp rx=0 tx=0).stop 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 -- 172.21.15.31:0/2277301610 shutdown_connections 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 --2- 172.21.15.31:0/2277301610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c102250 0x7f272c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 --2- 172.21.15.31:0/2277301610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f272c101890 0x7f272c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 --2- 172.21.15.31:0/2277301610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c1006a0 0x7f272c100ac0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 -- 172.21.15.31:0/2277301610 >> 172.21.15.31:0/2277301610 conn(0x7f272c0fbc30 msgr2=0x7f272c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 -- 172.21.15.31:0/2277301610 shutdown_connections 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.547+0000 7f2733163640 1 -- 172.21.15.31:0/2277301610 wait complete. 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 Processor -- start 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 -- start start 2024-04-25T00:29:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c1006a0 0x7f272c10b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 0x7f272c109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f272c102250 0x7f272c109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f272c10d310 con 0x7f272c101890 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f272c10d190 con 0x7f272c102250 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2733163640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f272c10d490 con 0x7f272c1006a0 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 0x7f272c109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 0x7f272c109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47672/0 (socket says 172.21.15.31:47672) 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 -- 172.21.15.31:0/436507584 learned_addr learned my addr 172.21.15.31:0/436507584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2730eff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c1006a0 0x7f272c10b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 -- 172.21.15.31:0/436507584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c1006a0 msgr2=0x7f272c10b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2731700640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f272c102250 0x7f272c109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c1006a0 0x7f272c10b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 -- 172.21.15.31:0/436507584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f272c102250 msgr2=0x7f272c109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f272c102250 0x7f272c109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2723fff640 1 -- 172.21.15.31:0/436507584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c10a430 con 0x7f272c101890 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.548+0000 7f2730eff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c1006a0 0x7f272c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.549+0000 7f2723fff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 0x7f272c109660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f271c00d950 tx=0x7f271c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.549+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c017070 con 0x7f272c101890 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.549+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f271c009c30 con 0x7f272c101890 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.549+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c1a1a90 con 0x7f272c101890 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.549+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c00f620 con 0x7f272c101890 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.549+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c1a1fe0 con 0x7f272c101890 2024-04-25T00:29:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.550+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c107a40 con 0x7f272c101890 2024-04-25T00:29:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.551+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f271c0109a0 con 0x7f272c101890 2024-04-25T00:29:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.552+0000 7f2721ffb640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27140a2b90 0x7f27140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.552+0000 7f2730eff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27140a2b90 0x7f27140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.552+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f271c0cedb0 con 0x7f272c101890 2024-04-25T00:29:54.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.552+0000 7f2730eff640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27140a2b90 0x7f27140a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f272802a930 tx=0x7f2728005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:54.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.556+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f271c0d3070 con 0x7f272c101890 2024-04-25T00:29:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.742+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f272c062340 con 0x7f272c101890 2024-04-25T00:29:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.744+0000 7f2721ffb640 1 -- 172.21.15.31:0/436507584 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22650 (secure 0 0 0) 0x7f271c009de0 con 0x7f272c101890 2024-04-25T00:29:54.745 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:54.745 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:29:14.426545+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:29:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27140a2b90 msgr2=0x7f27140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27140a2b90 0x7f27140a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f272802a930 tx=0x7f2728005fd0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 msgr2=0x7f272c109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 0x7f272c109660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f271c00d950 tx=0x7f271c00de10 comp rx=0 tx=0).stop 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 shutdown_connections 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f272c102250 0x7f272c109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f27140a2b90 0x7f27140a5050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c101890 0x7f272c109660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 --2- 172.21.15.31:0/436507584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272c1006a0 0x7f272c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 >> 172.21.15.31:0/436507584 conn(0x7f272c0fbc30 msgr2=0x7f272c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 shutdown_connections 2024-04-25T00:29:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.746+0000 7f2733163640 1 -- 172.21.15.31:0/436507584 wait complete. 2024-04-25T00:29:54.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:29:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 -- 172.21.15.31:0/609469536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 msgr2=0x7f313c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 --2- 172.21.15.31:0/609469536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c100ac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f312c009900 tx=0x7f312c02a410 comp rx=0 tx=0).stop 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 -- 172.21.15.31:0/609469536 shutdown_connections 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 --2- 172.21.15.31:0/609469536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c102250 0x7f313c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 --2- 172.21.15.31:0/609469536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c101890 0x7f313c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 --2- 172.21.15.31:0/609469536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c100ac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 -- 172.21.15.31:0/609469536 >> 172.21.15.31:0/609469536 conn(0x7f313c0fbc30 msgr2=0x7f313c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 -- 172.21.15.31:0/609469536 shutdown_connections 2024-04-25T00:29:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 -- 172.21.15.31:0/609469536 wait complete. 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.860+0000 7f3142215640 1 Processor -- start 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 -- start start 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c101890 0x7f313c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c102250 0x7f313c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f313c10c240 con 0x7f313c1006a0 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f313c10c0c0 con 0x7f313c101890 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f313c10c3c0 con 0x7f313c102250 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47684/0 (socket says 172.21.15.31:47684) 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 -- 172.21.15.31:0/2827998683 learned_addr learned my addr 172.21.15.31:0/2827998683 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313bfff640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c102250 0x7f313c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 -- 172.21.15.31:0/2827998683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c102250 msgr2=0x7f313c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c102250 0x7f313c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 -- 172.21.15.31:0/2827998683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c101890 msgr2=0x7f313c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313affd640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c101890 0x7f313c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c101890 0x7f313c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 -- 172.21.15.31:0/2827998683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c19f6e0 con 0x7f313c1006a0 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313affd640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c101890 0x7f313c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f313b7fe640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c197ac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f312c0314e0 tx=0x7f312c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c03f030 con 0x7f313c1006a0 2024-04-25T00:29:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c19f970 con 0x7f313c1006a0 2024-04-25T00:29:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f312c003cb0 con 0x7f313c1006a0 2024-04-25T00:29:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c035070 con 0x7f313c1006a0 2024-04-25T00:29:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.861+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c19fee0 con 0x7f313c1006a0 2024-04-25T00:29:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.862+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c1a00b0 con 0x7f313c1006a0 2024-04-25T00:29:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.863+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f312c042030 con 0x7f313c1006a0 2024-04-25T00:29:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.864+0000 7f3138ff9640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f31240a2b90 0x7f31240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.864+0000 7f313affd640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f31240a2b90 0x7f31240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.864+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f312c0ef740 con 0x7f313c1006a0 2024-04-25T00:29:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.865+0000 7f313affd640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f31240a2b90 0x7f31240a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f313c198fd0 tx=0x7f3130005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:54.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:54.869+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f312c0bc690 con 0x7f313c1006a0 2024-04-25T00:29:55.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:54 smithi031 ceph-mon[29167]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T00:29:55.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3557827172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:29:55.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4204288362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:55.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/436507584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:55.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.091+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f313c1a0440 con 0x7f313c1006a0 2024-04-25T00:29:55.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.092+0000 7f3138ff9640 1 -- 172.21.15.31:0/2827998683 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11926 (secure 0 0 0) 0x7f312c0bbf90 con 0x7f313c1006a0 2024-04-25T00:29:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:29:55.093 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:28:30.181657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24512,24517,24499,24434,24424]},"id":1}]} 2024-04-25T00:29:55.093 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f31240a2b90 msgr2=0x7f31240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f31240a2b90 0x7f31240a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f313c198fd0 tx=0x7f3130005e50 comp rx=0 tx=0).stop 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 msgr2=0x7f313c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c197ac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f312c0314e0 tx=0x7f312c0042e0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 shutdown_connections 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c102250 0x7f313c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f313c101890 0x7f313c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f31240a2b90 0x7f31240a5050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 --2- 172.21.15.31:0/2827998683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c1006a0 0x7f313c197ac0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.094+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 >> 172.21.15.31:0/2827998683 conn(0x7f313c0fbc30 msgr2=0x7f313c069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.095+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 shutdown_connections 2024-04-25T00:29:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.095+0000 7f3142215640 1 -- 172.21.15.31:0/2827998683 wait complete. 2024-04-25T00:29:55.107 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:28:23.753095+0000', 'modified': '2024-04-25T00:28:30.181657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24517, 'mds_1': 24512, 'mds_2': 24499, 'mds_3': 24434, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/2045809592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 2045809592}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 2045809592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/3504624607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3504624607}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3504624607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.169:6833/2672070962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 2672070962}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 2672070962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'j', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.31:6835/310214957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 310214957}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 310214957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/653343360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 653343360}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 653343360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6837/3765244618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6836', 'nonce': 3765244618}, {'type': 'v1', 'addr': '172.21.15.40:6837', 'nonce': 3765244618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/3258138877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3258138877}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3258138877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6837/1684471092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1684471092}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1684471092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6839/2724938586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 2724938586}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 2724938586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.169:6839/3109017383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 3109017383}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 3109017383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24517, 'qdb_cluster': [24512, 24517, 24499, 24434, 24424]} 2024-04-25T00:29:55.107 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T00:29:55.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-25T00:29:55.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:54 smithi169 ceph-mon[33112]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T00:29:55.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3557827172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:29:55.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4204288362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:55.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/436507584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:54 smithi040 ceph-mon[32945]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T00:29:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3557827172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:29:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4204288362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/436507584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:29:55.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 -- 172.21.15.31:0/2974881285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb90107520 msgr2=0x7feb90109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 --2- 172.21.15.31:0/2974881285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb90107520 0x7feb90109910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7feb88009ca0 tx=0x7feb8802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 -- 172.21.15.31:0/2974881285 shutdown_connections 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 --2- 172.21.15.31:0/2974881285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb90107520 0x7feb90109910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 --2- 172.21.15.31:0/2974881285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb90104bf0 0x7feb90106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 --2- 172.21.15.31:0/2974881285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb901022c0 0x7feb901046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 -- 172.21.15.31:0/2974881285 >> 172.21.15.31:0/2974881285 conn(0x7feb900fbc10 msgr2=0x7feb900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 -- 172.21.15.31:0/2974881285 shutdown_connections 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.210+0000 7feb977f4640 1 -- 172.21.15.31:0/2974881285 wait complete. 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 Processor -- start 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 -- start start 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 0x7feb901979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb90104bf0 0x7feb90197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb90107520 0x7feb9019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010d2d0 con 0x7feb901022c0 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010d150 con 0x7feb90107520 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb977f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010d450 con 0x7feb90104bf0 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 0x7feb901979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 0x7feb901979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47702/0 (socket says 172.21.15.31:47702) 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 -- 172.21.15.31:0/3339685422 learned_addr learned my addr 172.21.15.31:0/3339685422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb94d8f640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb90104bf0 0x7feb90197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 -- 172.21.15.31:0/3339685422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb90104bf0 msgr2=0x7feb90197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb90104bf0 0x7feb90197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 -- 172.21.15.31:0/3339685422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb90107520 msgr2=0x7feb9019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95d91640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb90107520 0x7feb9019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb90107520 0x7feb9019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 -- 172.21.15.31:0/3339685422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9019f5f0 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95d91640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb90107520 0x7feb9019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.211+0000 7feb95590640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 0x7feb901979d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00d950 tx=0x7feb8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.212+0000 7feb867fc640 1 -- 172.21.15.31:0/3339685422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c017070 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.212+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9019f8e0 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.212+0000 7feb867fc640 1 -- 172.21.15.31:0/3339685422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c009c30 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.212+0000 7feb867fc640 1 -- 172.21.15.31:0/3339685422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00f620 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.212+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9019fe80 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.214+0000 7feb867fc640 1 -- 172.21.15.31:0/3339685422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb8c0109a0 con 0x7feb901022c0 2024-04-25T00:29:55.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.214+0000 7feb867fc640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7feb7c0a2b90 0x7feb7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.214+0000 7feb94d8f640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7feb7c0a2b90 0x7feb7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.214+0000 7feb867fc640 1 -- 172.21.15.31:0/3339685422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7feb8c0ced50 con 0x7feb901022c0 2024-04-25T00:29:55.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 Processor -- start 2024-04-25T00:29:55.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb94d8f640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7feb7c0a2b90 0x7feb7c0a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb90198ee0 tx=0x7feb8000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 -- start start 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a66930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eaf59c4f00 con 0x55eaf5a61e30 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55eaf596d750 con 0x55eaf5a61440 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eaf59c0ec0 con 0x55eaf5a66fa0 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb94d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a66930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb94d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a66930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47708/0 (socket says 172.21.15.31:47708) 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb94d8f640 1 -- 172.21.15.31:0/786944871 learned_addr learned my addr 172.21.15.31:0/786944871 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.215+0000 7feb95590640 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb95d91640 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb94d8f640 1 -- 172.21.15.31:0/786944871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 msgr2=0x55eaf5a69430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb94d8f640 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a69430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb94d8f640 1 -- 172.21.15.31:0/786944871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 msgr2=0x55eaf5a61860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb94d8f640 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a61860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb94d8f640 1 -- 172.21.15.31:0/786944871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf56d92d0 con 0x55eaf5a61e30 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb95d91640 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a69430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb94d8f640 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a66930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feb8000edb0 tx=0x7feb80011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970e61890613b317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb7b7fe640 1 -- 172.21.15.31:0/786944871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80021070 con 0x55eaf5a61e30 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb7b7fe640 1 -- 172.21.15.31:0/786944871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8000a740 con 0x55eaf5a61e30 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.216+0000 7feb7b7fe640 1 -- 172.21.15.31:0/786944871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8000fa60 con 0x55eaf5a61e30 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 -- 172.21.15.31:0/786944871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 msgr2=0x55eaf5a66930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a66930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feb8000edb0 tx=0x7feb80011000 comp rx=0 tx=0).stop 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 -- 172.21.15.31:0/786944871 shutdown_connections 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a69430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a66930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/786944871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 -- 172.21.15.31:0/786944871 >> 172.21.15.31:0/786944871 conn(0x55eaf5a5cab0 msgr2=0x55eaf5a5ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 -- 172.21.15.31:0/786944871 shutdown_connections 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 -- 172.21.15.31:0/786944871 wait complete. 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 -- start start 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.217+0000 7feb9a5d7740 1 Processor -- start 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 0x55eaf5a6b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a69f50 con 0x55eaf5a61440 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5980cf0 con 0x55eaf5a66fa0 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a6a0d0 con 0x55eaf5a61e30 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 0x55eaf5a6b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 10 client.0 _ll_register_callbacks cb 0x55eaf59806e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 0x55eaf5a6b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47722/0 (socket says 172.21.15.31:47722) 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 -- 172.21.15.31:0/2246436556 learned_addr learned my addr 172.21.15.31:0/2246436556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95d91640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 -- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61e30 msgr2=0x55eaf5a6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 -- 172.21.15.31:0/2246436556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 msgr2=0x55eaf5a72de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 -- 172.21.15.31:0/2246436556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf57a36b0 con 0x55eaf5a61440 2024-04-25T00:29:55.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95d91640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb95590640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 0x55eaf5a6b860 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0cc840 tx=0x7feb8c009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394de87eee7eefca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb78ff9640 1 -- 172.21.15.31:0/2246436556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c0b4060 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb78ff9640 1 -- 172.21.15.31:0/2246436556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7feb8c097c10 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8000a740 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb78ff9640 1 -- 172.21.15.31:0/2246436556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb8c020040 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 10 client.15069 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.218+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eaf5a7a210 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb78ff9640 1 -- 172.21.15.31:0/2246436556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8000a740 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb78ff9640 1 -- 172.21.15.31:0/2246436556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feb8c09ab80 con 0x55eaf5a61440 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb9a5d7740 10 client.15069 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb9a5d7740 10 client.15069 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T00:29:55.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb9a5d7740 20 client.15069 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5a7c8f0 0x55eaf5a7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb9a5d7740 4 client.15069 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 --> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaf59d2440 con 0x55eaf5a7c8f0 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.219+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5a7c8f0 0x55eaf5a7dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.220+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5a7c8f0 0x55eaf5a7dd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=924e21c8c5e200b8 server_cookie=64e8cf00cd3ac765 in_seq=0 out_seq=0 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.220+0000 7feb78ff9640 10 client.15069 ms_handle_connect on v2:172.21.15.169:6832/2672070962 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.220+0000 7feb78ff9640 1 -- 172.21.15.31:0/2246436556 <== mds.4 v2:172.21.15.169:6832/2672070962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eaf59d2440 con 0x55eaf5a7c8f0 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.220+0000 7feb78ff9640 10 client.15069 handle_command_reply: tid=0 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.220+0000 7feb9a5d7740 1 client.15069 shutdown 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 msgr2=0x55eaf5a6b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 0x55eaf5a6b860 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0cc840 tx=0x7feb8c009de0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 shutdown_connections 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5a7c8f0 0x55eaf5a7dd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb78ff9640 0 client.15069 ms_handle_reset on v2:172.21.15.169:6832/2672070962 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2246436556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61440 0x55eaf5a6b860 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 >> 172.21.15.31:0/2246436556 conn(0x55eaf5a5cab0 msgr2=0x55eaf5a5bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 shutdown_connections 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2246436556 wait complete. 2024-04-25T00:29:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.221+0000 7feb9a5d7740 20 client.15069 trim_cache size 0 max 16384 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 Processor -- start 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 -- start start 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61440 0x55eaf5a5bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a7c0a0 con 0x55eaf5a61e30 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a7c770 con 0x55eaf5a66fa0 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a7ce40 con 0x55eaf5a61440 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb94d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb94d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47732/0 (socket says 172.21.15.31:47732) 2024-04-25T00:29:55.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.239+0000 7feb94d8f640 1 -- 172.21.15.31:0/1043844827 learned_addr learned my addr 172.21.15.31:0/1043844827 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb94d8f640 1 -- 172.21.15.31:0/1043844827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61440 msgr2=0x55eaf5a5bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb94d8f640 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61440 0x55eaf5a5bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb94d8f640 1 -- 172.21.15.31:0/1043844827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 msgr2=0x55eaf5a72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb94d8f640 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb94d8f640 1 -- 172.21.15.31:0/1043844827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf5a664a0 con 0x55eaf5a61e30 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb94d8f640 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6a4d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb8000edb0 tx=0x7feb80010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af6834fb074a4e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb7affd640 1 -- 172.21.15.31:0/1043844827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80014040 con 0x55eaf5a61e30 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb7affd640 1 -- 172.21.15.31:0/1043844827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb800053c0 con 0x55eaf5a61e30 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.240+0000 7feb7affd640 1 -- 172.21.15.31:0/1043844827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80021040 con 0x55eaf5a61e30 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- 172.21.15.31:0/1043844827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 msgr2=0x55eaf5a6a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6a4d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb8000edb0 tx=0x7feb80010dc0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- 172.21.15.31:0/1043844827 shutdown_connections 2024-04-25T00:29:55.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6a4d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/1043844827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a61440 0x55eaf5a5bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- 172.21.15.31:0/1043844827 >> 172.21.15.31:0/1043844827 conn(0x55eaf5a5cab0 msgr2=0x55eaf5a57690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- 172.21.15.31:0/1043844827 shutdown_connections 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- 172.21.15.31:0/1043844827 wait complete. 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- start start 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 Processor -- start 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a6ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a71e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a7dbe0 con 0x55eaf5a61e30 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a7d510 con 0x55eaf5a61440 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eaf5a7e2b0 con 0x55eaf5a66fa0 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb9a5d7740 10 client.0 _ll_register_callbacks cb 0x7feb8c09b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.241+0000 7feb94d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47744/0 (socket says 172.21.15.31:47744) 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 -- 172.21.15.31:0/2729238042 learned_addr learned my addr 172.21.15.31:0/2729238042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 -- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 msgr2=0x55eaf5a71e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a71e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 -- 172.21.15.31:0/2729238042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 msgr2=0x55eaf5a6ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a6ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 -- 172.21.15.31:0/2729238042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf5a72470 con 0x55eaf5a61e30 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb94d8f640 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6f470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb8000f010 tx=0x7feb8000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367cec332cfb5c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb78ff9640 1 -- 172.21.15.31:0/2729238042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80021040 con 0x55eaf5a61e30 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb78ff9640 1 -- 172.21.15.31:0/2729238042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7feb8000f6d0 con 0x55eaf5a61e30 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb800053c0 con 0x55eaf5a61e30 2024-04-25T00:29:55.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb78ff9640 1 -- 172.21.15.31:0/2729238042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb8000f8d0 con 0x55eaf5a61e30 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb9a5d7740 10 client.15075 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eaf5ae2d00 con 0x55eaf5a61e30 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb78ff9640 1 -- 172.21.15.31:0/2729238042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb800053c0 con 0x55eaf5a61e30 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.242+0000 7feb78ff9640 1 -- 172.21.15.31:0/2729238042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feb80004660 con 0x55eaf5a61e30 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 10 client.15075 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 10 client.15075 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 20 client.15075 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5ae7350 0x55eaf5ae6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 4 client.15075 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 --> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] -- command(tid 0: {"prefix": "status"}) -- 0x55eaf59cc790 con 0x55eaf5ae7350 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb95590640 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5ae7350 0x55eaf5ae6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb78ff9640 10 client.15075 ms_handle_connect on v2:172.21.15.169:6832/2672070962 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb95590640 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5ae7350 0x55eaf5ae6060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29cb367d91a0a6f2 server_cookie=1bd1e5e8f3530e8a in_seq=0 out_seq=0 2024-04-25T00:29:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb78ff9640 1 -- 172.21.15.31:0/2729238042 <== mds.4 v2:172.21.15.169:6832/2672070962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55eaf59cc790 con 0x55eaf5ae7350 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb78ff9640 10 client.15075 handle_command_reply: tid=0 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.243+0000 7feb9a5d7740 1 client.15075 shutdown 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 msgr2=0x55eaf5a6f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6f470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb8000f010 tx=0x7feb8000fc10 comp rx=0 tx=0).stop 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 shutdown_connections 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eaf5a66fa0 0x55eaf5a71e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaf5a61e30 0x55eaf5a6f470 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x55eaf5ae7350 0x55eaf5ae6060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 --2- 172.21.15.31:0/2729238042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaf5a61440 0x55eaf5a6ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb78ff9640 0 client.15075 ms_handle_reset on v2:172.21.15.169:6832/2672070962 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 >> 172.21.15.31:0/2729238042 conn(0x55eaf5a5cab0 msgr2=0x55eaf5a666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 shutdown_connections 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 1 -- 172.21.15.31:0/2729238042 wait complete. 2024-04-25T00:29:55.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb9a5d7740 20 client.15075 trim_cache size 0 max 16384 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24424, 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 86.078671206999999, 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T00:29:55.251 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:29:55.252 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 129.265877168 2024-04-25T00:29:55.252 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7feb7c0a2b90 msgr2=0x7feb7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7feb7c0a2b90 0x7feb7c0a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb90198ee0 tx=0x7feb8000a4c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 msgr2=0x7feb901979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 0x7feb901979d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00d950 tx=0x7feb8c00de10 comp rx=0 tx=0).stop 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 shutdown_connections 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb90107520 0x7feb9019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb90104bf0 0x7feb90197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7feb7c0a2b90 0x7feb7c0a5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 --2- 172.21.15.31:0/3339685422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb901022c0 0x7feb901979d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 >> 172.21.15.31:0/3339685422 conn(0x7feb900fbc10 msgr2=0x7feb90102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 shutdown_connections 2024-04-25T00:29:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.244+0000 7feb977f4640 1 -- 172.21.15.31:0/3339685422 wait complete. 2024-04-25T00:29:55.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.j status 2024-04-25T00:29:55.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 -- 172.21.15.31:0/2929244926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 msgr2=0x7fbc38100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/2929244926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc38100ac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009900 tx=0x7fbc2802a410 comp rx=0 tx=0).stop 2024-04-25T00:29:55.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 -- 172.21.15.31:0/2929244926 shutdown_connections 2024-04-25T00:29:55.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/2929244926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc38102250 0x7fbc38108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/2929244926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc38101890 0x7fbc38101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/2929244926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc38100ac0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 -- 172.21.15.31:0/2929244926 >> 172.21.15.31:0/2929244926 conn(0x7fbc380fbc30 msgr2=0x7fbc380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 -- 172.21.15.31:0/2929244926 shutdown_connections 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 -- 172.21.15.31:0/2929244926 wait complete. 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.358+0000 7fbc3d07b640 1 Processor -- start 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 -- start start 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc3806a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc38101890 0x7fbc3819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc38102250 0x7fbc381a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3810c2f0 con 0x7fbc381006a0 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3810c170 con 0x7fbc38102250 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3810c470 con 0x7fbc38101890 2024-04-25T00:29:55.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc3806a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc3806a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47766/0 (socket says 172.21.15.31:47766) 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 -- 172.21.15.31:0/3933288738 learned_addr learned my addr 172.21.15.31:0/3933288738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3659c640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc38101890 0x7fbc3819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3759e640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc38102250 0x7fbc381a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 -- 172.21.15.31:0/3933288738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc38101890 msgr2=0x7fbc3819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc38101890 0x7fbc3819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 -- 172.21.15.31:0/3933288738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc38102250 msgr2=0x7fbc381a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc38102250 0x7fbc381a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 -- 172.21.15.31:0/3933288738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc381a19d0 con 0x7fbc381006a0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc3806a860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbc280314e0 tx=0x7fbc28003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc1ffff640 1 -- 172.21.15.31:0/3933288738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2803f070 con 0x7fbc381006a0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.359+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc381a1c60 con 0x7fbc381006a0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.360+0000 7fbc1ffff640 1 -- 172.21.15.31:0/3933288738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc28031e40 con 0x7fbc381006a0 2024-04-25T00:29:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.360+0000 7fbc1ffff640 1 -- 172.21.15.31:0/3933288738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28035030 con 0x7fbc381006a0 2024-04-25T00:29:55.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.360+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc381a21a0 con 0x7fbc381006a0 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.362+0000 7fbc1ffff640 1 -- 172.21.15.31:0/3933288738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc28042070 con 0x7fbc381006a0 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.362+0000 7fbc3fd82740 1 Processor -- start 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 -- start start 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87efc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efce30 0x55cf87f01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 0x55cf87f04430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e5ff00 con 0x55cf87f01fa0 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e08750 con 0x55cf87efc440 2024-04-25T00:29:55.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e5bec0 con 0x55cf87efce30 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 0x55cf87f04430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 0x55cf87f04430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47772/0 (socket says 172.21.15.31:47772) 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 -- 172.21.15.31:0/2540877256 learned_addr learned my addr 172.21.15.31:0/2540877256 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc1ffff640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbc200a2bc0 0x7fbc200a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efce30 0x55cf87f01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3759e640 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87efc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 -- 172.21.15.31:0/2540877256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efce30 msgr2=0x55cf87f01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efce30 0x55cf87f01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 -- 172.21.15.31:0/2540877256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 msgr2=0x55cf87efc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87efc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 -- 172.21.15.31:0/2540877256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf87b742d0 con 0x55cf87f01fa0 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3759e640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbc200a2bc0 0x7fbc200a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efce30 0x55cf87f01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:55.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc1ffff640 1 -- 172.21.15.31:0/3933288738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbc280efad0 con 0x7fbc381006a0 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.363+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 0x55cf87f04430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00b7c0 tx=0x7fbc2c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4760f142bd7c1489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc1d7fa640 1 -- 172.21.15.31:0/2540877256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00d950 con 0x55cf87f01fa0 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc1d7fa640 1 -- 172.21.15.31:0/2540877256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2c018070 con 0x55cf87f01fa0 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc1d7fa640 1 -- 172.21.15.31:0/2540877256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0123e0 con 0x55cf87f01fa0 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2540877256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 msgr2=0x55cf87f04430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 0x55cf87f04430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00b7c0 tx=0x7fbc2c00bc80 comp rx=0 tx=0).stop 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3759e640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbc200a2bc0 0x7fbc200a5080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc300070b0 tx=0x7fbc3000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2540877256 shutdown_connections 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87f01fa0 0x55cf87f04430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efce30 0x55cf87f01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2540877256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87efc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2540877256 >> 172.21.15.31:0/2540877256 conn(0x55cf87ef7ab0 msgr2=0x55cf87ef7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2540877256 shutdown_connections 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.364+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2540877256 wait complete. 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 -- start start 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 Processor -- start 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 0x55cf87f06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efce30 0x55cf87f06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f04f50 con 0x55cf87efc440 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e1bcf0 con 0x55cf87efce30 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f050d0 con 0x55cf87f01fa0 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 0x55cf87f06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc3fd82740 10 client.0 _ll_register_callbacks cb 0x55cf87e1b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 0x55cf87f06820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47778/0 (socket says 172.21.15.31:47778) 2024-04-25T00:29:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 -- 172.21.15.31:0/2050547732 learned_addr learned my addr 172.21.15.31:0/2050547732 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 -- 172.21.15.31:0/2050547732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 msgr2=0x55cf87f0dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 -- 172.21.15.31:0/2050547732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efce30 msgr2=0x55cf87f06d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efce30 0x55cf87f06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 -- 172.21.15.31:0/2050547732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf87c3e6b0 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 0x55cf87f06820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc280c4a60 tx=0x7fbc280bbf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b952aa29defb796d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.365+0000 7fbc1aff5640 1 -- 172.21.15.31:0/2050547732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28049040 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc1aff5640 1 -- 172.21.15.31:0/2050547732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbc2803b460 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c018070 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc1aff5640 1 -- 172.21.15.31:0/2050547732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc280c06d0 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 10 client.15087 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf87f10cf0 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc1aff5640 1 -- 172.21.15.31:0/2050547732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2c018070 con 0x55cf87efc440 2024-04-25T00:29:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc1aff5640 1 -- 172.21.15.31:0/2050547732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbc280b8990 con 0x55cf87efc440 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 10 client.15087 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 10 client.15087 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 20 client.15087 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f13480 0x55cf87f148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 4 client.15087 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 --> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf87e6d440 con 0x55cf87f13480 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f13480 0x55cf87f148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f13480 0x55cf87f148f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c12d2afa740fb0e7 server_cookie=48abaabcd9ffe62 in_seq=0 out_seq=0 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.366+0000 7fbc1aff5640 10 client.15087 ms_handle_connect on v2:172.21.15.31:6834/310214957 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc1aff5640 1 -- 172.21.15.31:0/2050547732 <== mds.3 v2:172.21.15.31:6834/310214957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf87e6d440 con 0x55cf87f13480 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc1aff5640 10 client.15087 handle_command_reply: tid=0 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 client.15087 shutdown 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 msgr2=0x55cf87f06820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 0x55cf87f06820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc280c4a60 tx=0x7fbc280bbf20 comp rx=0 tx=0).stop 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 shutdown_connections 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f13480 0x55cf87f148f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efce30 0x55cf87f06d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2050547732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efc440 0x55cf87f06820 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc1aff5640 0 client.15087 ms_handle_reset on v2:172.21.15.31:6834/310214957 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 >> 172.21.15.31:0/2050547732 conn(0x55cf87ef7ab0 msgr2=0x55cf87ef6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 shutdown_connections 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2050547732 wait complete. 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.367+0000 7fbc3fd82740 20 client.15087 trim_cache size 0 max 16384 2024-04-25T00:29:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.385+0000 7fbc3fd82740 1 Processor -- start 2024-04-25T00:29:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.385+0000 7fbc3fd82740 1 -- start start 2024-04-25T00:29:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87f06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f06970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f12c30 con 0x55cf87efce30 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f13300 con 0x55cf87efc440 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f139d0 con 0x55cf87f01fa0 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47782/0 (socket says 172.21.15.31:47782) 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 -- 172.21.15.31:0/2583302585 learned_addr learned my addr 172.21.15.31:0/2583302585 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3759e640 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f06970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 -- 172.21.15.31:0/2583302585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 msgr2=0x55cf87f06970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f06970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87f06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 -- 172.21.15.31:0/2583302585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 msgr2=0x55cf87f06430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87f06430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 -- 172.21.15.31:0/2583302585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf87efd210 con 0x55cf87efce30 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3759e640 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f06970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:55.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc3659c640 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0dd60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c015040 tx=0x7fbc2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322cf6eb835a9957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc1cff9640 1 -- 172.21.15.31:0/2583302585 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c023d50 con 0x55cf87efce30 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc1cff9640 1 -- 172.21.15.31:0/2583302585 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2c018040 con 0x55cf87efce30 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.386+0000 7fbc1cff9640 1 -- 172.21.15.31:0/2583302585 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c023600 con 0x55cf87efce30 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2583302585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 msgr2=0x55cf87f0dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0dd60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c015040 tx=0x7fbc2c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2583302585 shutdown_connections 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87f01fa0 0x55cf87f06970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0dd60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/2583302585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87efc440 0x55cf87f06430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2583302585 >> 172.21.15.31:0/2583302585 conn(0x55cf87ef7ab0 msgr2=0x55cf87efd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2583302585 shutdown_connections 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 -- 172.21.15.31:0/2583302585 wait complete. 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 -- start start 2024-04-25T00:29:55.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.387+0000 7fbc3fd82740 1 Processor -- start 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efc440 0x55cf87f09f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87f01fa0 0x55cf87f0ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f14770 con 0x55cf87efce30 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f140a0 con 0x55cf87f01fa0 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cf87f14e40 con 0x55cf87efc440 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 10 client.0 _ll_register_callbacks cb 0x7fbc280bc0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47798/0 (socket says 172.21.15.31:47798) 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 -- 172.21.15.31:0/3194838709 learned_addr learned my addr 172.21.15.31:0/3194838709 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efc440 0x55cf87f09f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3759e640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87f01fa0 0x55cf87f0ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 -- 172.21.15.31:0/3194838709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efc440 msgr2=0x55cf87f09f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efc440 0x55cf87f09f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 -- 172.21.15.31:0/3194838709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87f01fa0 msgr2=0x55cf87f0ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87f01fa0 0x55cf87f0ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 -- 172.21.15.31:0/3194838709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf87f0d510 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3659c640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0a4a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00df80 tx=0x7fbc2c014d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fedd3a30e0d512c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc1aff5640 1 -- 172.21.15.31:0/3194838709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c028070 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc1aff5640 1 -- 172.21.15.31:0/3194838709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbc2c013790 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c018040 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc1aff5640 1 -- 172.21.15.31:0/3194838709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc2c00da90 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 10 client.15093 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.388+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf87f79860 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc1aff5640 1 -- 172.21.15.31:0/3194838709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2c018040 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc1aff5640 1 -- 172.21.15.31:0/3194838709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbc2c02b3f0 con 0x55cf87efce30 2024-04-25T00:29:55.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 10 client.15093 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 10 client.15093 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 20 client.15093 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f7dca0 0x55cf87f7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 4 client.15093 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 --> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] -- command(tid 0: {"prefix": "status"}) -- 0x55cf87e67790 con 0x55cf87f7dca0 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f7dca0 0x55cf87f7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc1aff5640 10 client.15093 ms_handle_connect on v2:172.21.15.31:6834/310214957 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc36d9d640 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f7dca0 0x55cf87f7c9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d453170b64d2f61b server_cookie=3665c45dcce81ba in_seq=0 out_seq=0 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc1aff5640 1 -- 172.21.15.31:0/3194838709 <== mds.3 v2:172.21.15.31:6834/310214957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cf87e67790 con 0x55cf87f7dca0 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc1aff5640 10 client.15093 handle_command_reply: tid=0 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.389+0000 7fbc3fd82740 1 client.15093 shutdown 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 msgr2=0x55cf87f0a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0a4a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00df80 tx=0x7fbc2c014d80 comp rx=0 tx=0).stop 2024-04-25T00:29:55.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 shutdown_connections 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x55cf87f7dca0 0x55cf87f7c9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf87f01fa0 0x55cf87f0ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cf87efce30 0x55cf87f0a4a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 --2- 172.21.15.31:0/3194838709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cf87efc440 0x55cf87f09f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc1aff5640 0 client.15093 ms_handle_reset on v2:172.21.15.31:6834/310214957 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 >> 172.21.15.31:0/3194838709 conn(0x55cf87ef7ab0 msgr2=0x55cf87efa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 shutdown_connections 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 1 -- 172.21.15.31:0/3194838709 wait complete. 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3fd82740 20 client.15093 trim_cache size 0 max 16384 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:29:55.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24434, 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 87.229894306000006, 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 123.563553029 2024-04-25T00:29:55.398 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbc200a2bc0 msgr2=0x7fbc200a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbc200a2bc0 0x7fbc200a5080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc300070b0 tx=0x7fbc3000c040 comp rx=0 tx=0).stop 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 msgr2=0x7fbc3806a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc3806a860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbc280314e0 tx=0x7fbc28003d50 comp rx=0 tx=0).stop 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 shutdown_connections 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc38102250 0x7fbc381a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc38101890 0x7fbc3819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fbc200a2bc0 0x7fbc200a5080 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 --2- 172.21.15.31:0/3933288738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc381006a0 0x7fbc3806a860 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 >> 172.21.15.31:0/3933288738 conn(0x7fbc380fbc30 msgr2=0x7fbc38104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 shutdown_connections 2024-04-25T00:29:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.390+0000 7fbc3d07b640 1 -- 172.21.15.31:0/3933288738 wait complete. 2024-04-25T00:29:55.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:29:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 -- 172.21.15.31:0/3443224741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 msgr2=0x7f7cf0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/3443224741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf0108ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009ca0 tx=0x7f7ce802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 -- 172.21.15.31:0/3443224741 shutdown_connections 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/3443224741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf0108ae0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/3443224741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cf0101890 0x7f7cf0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/3443224741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf01006a0 0x7f7cf0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 -- 172.21.15.31:0/3443224741 >> 172.21.15.31:0/3443224741 conn(0x7f7cf00fbc30 msgr2=0x7f7cf00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.503+0000 7f7cf68d4640 1 -- 172.21.15.31:0/3443224741 shutdown_connections 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 -- 172.21.15.31:0/3443224741 wait complete. 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 Processor -- start 2024-04-25T00:29:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 -- start start 2024-04-25T00:29:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf01006a0 0x7f7cf01967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cf0101890 0x7f7cf0196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf010c1b0 con 0x7f7cf0102250 2024-04-25T00:29:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf010c030 con 0x7f7cf01006a0 2024-04-25T00:29:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.504+0000 7f7cf68d4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf010c330 con 0x7f7cf0101890 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf019dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47820/0 (socket says 172.21.15.31:47820) 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 -- 172.21.15.31:0/1601917513 learned_addr learned my addr 172.21.15.31:0/1601917513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7ceffff640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf01006a0 0x7f7cf01967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cf0101890 0x7f7cf0196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 -- 172.21.15.31:0/1601917513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cf0101890 msgr2=0x7f7cf0196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cf0101890 0x7f7cf0196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 -- 172.21.15.31:0/1601917513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf01006a0 msgr2=0x7f7cf01967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf01006a0 0x7f7cf01967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 -- 172.21.15.31:0/1601917513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf019e3e0 con 0x7f7cf0102250 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf4e71640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf019dd70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009dc0 tx=0x7f7ce8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7ced7fa640 1 -- 172.21.15.31:0/1601917513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8032ec0 con 0x7f7cf0102250 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7ced7fa640 1 -- 172.21.15.31:0/1601917513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ce80044f0 con 0x7f7cf0102250 2024-04-25T00:29:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7ced7fa640 1 -- 172.21.15.31:0/1601917513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8030d40 con 0x7f7cf0102250 2024-04-25T00:29:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf019e670 con 0x7f7cf0102250 2024-04-25T00:29:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.505+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf019eb70 con 0x7f7cf0102250 2024-04-25T00:29:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.507+0000 7f7ced7fa640 1 -- 172.21.15.31:0/1601917513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ce8004040 con 0x7f7cf0102250 2024-04-25T00:29:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 Processor -- start 2024-04-25T00:29:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 -- start start 2024-04-25T00:29:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 0x55ac4171d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4171dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41729b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac41680f00 con 0x55ac4171eac0 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ac41629750 con 0x55ac4171d750 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ac4167cec0 con 0x55ac41727740 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7ced7fa640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7cd80a2d30 0x7f7cd80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 0x55ac4171d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 0x55ac4171d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47822/0 (socket says 172.21.15.31:47822) 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2181160099 learned_addr learned my addr 172.21.15.31:0/2181160099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cf4e71640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7cd80a2d30 0x7f7cd80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7ceffff640 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4171dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7ced7fa640 1 -- 172.21.15.31:0/1601917513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7ce80f0630 con 0x7f7cf0102250 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.508+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41729b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2181160099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 msgr2=0x55ac41729b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41729b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2181160099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 msgr2=0x55ac4171dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4171dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2181160099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac413952d0 con 0x55ac4171eac0 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cf4e71640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7cd80a2d30 0x7f7cd80a51f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80bcb80 tx=0x7f7ce80dd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 0x55ac4171d210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000bef0 tx=0x7f7ce000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1aadc8b0a69435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cd67fc640 1 -- 172.21.15.31:0/2181160099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0004030 con 0x55ac4171eac0 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cd67fc640 1 -- 172.21.15.31:0/2181160099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ce0019070 con 0x55ac4171eac0 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.509+0000 7f7cd67fc640 1 -- 172.21.15.31:0/2181160099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0003de0 con 0x55ac4171eac0 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2181160099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 msgr2=0x55ac4171d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 0x55ac4171d210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000bef0 tx=0x7f7ce000b670 comp rx=0 tx=0).stop 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2181160099 shutdown_connections 2024-04-25T00:29:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41729b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4171dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2181160099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171eac0 0x55ac4171d210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2181160099 >> 172.21.15.31:0/2181160099 conn(0x55ac41718ab0 msgr2=0x55ac41718eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2181160099 shutdown_connections 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2181160099 wait complete. 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- start start 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 Processor -- start 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4172bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 0x55ac41733200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac4172a370 con 0x55ac41727740 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ac4163ccf0 con 0x55ac4171d750 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ac4172a4f0 con 0x55ac4171eac0 2024-04-25T00:29:55.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.510+0000 7f7cf95e3740 10 client.0 _ll_register_callbacks cb 0x55ac4163c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 0x55ac41733200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 0x55ac41733200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47832/0 (socket says 172.21.15.31:47832) 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2646449439 learned_addr learned my addr 172.21.15.31:0/2646449439 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4172bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7ceffff640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 msgr2=0x55ac4172c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2646449439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 msgr2=0x55ac4172bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4172bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 -- 172.21.15.31:0/2646449439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4145f6b0 con 0x55ac41727740 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 0x55ac41733200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0012010 tx=0x7f7ce000b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5598555931b6d6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7ccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/2646449439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0003d90 con 0x55ac41727740 2024-04-25T00:29:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/2646449439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7ce0003f30 con 0x55ac41727740 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce0019070 con 0x55ac41727740 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/2646449439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ce0004130 con 0x55ac41727740 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cf95e3740 10 client.15105 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac41736250 con 0x55ac41727740 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.511+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/2646449439 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ce0019070 con 0x55ac41727740 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/2646449439 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7ce001f410 con 0x55ac41727740 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7cf95e3740 10 client.15105 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7cf95e3740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7cf95e3740 20 client.15105 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417386a0 0x55ac41739b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7cf95e3740 4 client.15105 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 --> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac4168e440 con 0x55ac417386a0 2024-04-25T00:29:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7ceffff640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417386a0 0x55ac41739b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7ccf7fe640 10 client.15105 ms_handle_connect on v2:172.21.15.169:6836/1684471092 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.512+0000 7f7ceffff640 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417386a0 0x55ac41739b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7456ae637c4d12d7 server_cookie=3b1ae9ee5b7d6a5e in_seq=0 out_seq=0 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/2646449439 <== mds.2 v2:172.21.15.169:6836/1684471092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac4168e440 con 0x55ac417386a0 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7ccf7fe640 10 client.15105 handle_command_reply: tid=0 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 client.15105 shutdown 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 msgr2=0x55ac41733200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 0x55ac41733200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0012010 tx=0x7f7ce000b5d0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 shutdown_connections 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417386a0 0x55ac41739b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac41727740 0x55ac41733200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7ccf7fe640 0 client.15105 ms_handle_reset on v2:172.21.15.169:6836/1684471092 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/2646449439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171d750 0x55ac4172bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.513+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 >> 172.21.15.31:0/2646449439 conn(0x55ac41718ab0 msgr2=0x55ac41717ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.514+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 shutdown_connections 2024-04-25T00:29:55.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.514+0000 7f7cf95e3740 1 -- 172.21.15.31:0/2646449439 wait complete. 2024-04-25T00:29:55.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.514+0000 7f7cf95e3740 20 client.15105 trim_cache size 0 max 16384 2024-04-25T00:29:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 Processor -- start 2024-04-25T00:29:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 -- start start 2024-04-25T00:29:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4171cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171eac0 0x55ac41733120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41718e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac41737e50 con 0x55ac4171d750 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ac41738520 con 0x55ac4171eac0 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ac41738bf0 con 0x55ac41727740 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4171cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4171cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47848/0 (socket says 172.21.15.31:47848) 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1491169922 learned_addr learned my addr 172.21.15.31:0/1491169922 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1491169922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 msgr2=0x55ac41718e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41718e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1491169922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171eac0 msgr2=0x55ac41733120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171eac0 0x55ac41733120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1491169922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac41716f70 con 0x55ac4171d750 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4171cfa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0000c00 tx=0x7f7ce000b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a19ead94eab8bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cd5ffb640 1 -- 172.21.15.31:0/1491169922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0023020 con 0x55ac4171d750 2024-04-25T00:29:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cd5ffb640 1 -- 172.21.15.31:0/1491169922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ce0019070 con 0x55ac4171d750 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.532+0000 7f7cd5ffb640 1 -- 172.21.15.31:0/1491169922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0004470 con 0x55ac4171d750 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1491169922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 msgr2=0x55ac4171cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4171cfa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0000c00 tx=0x7f7ce000b4d0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1491169922 shutdown_connections 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac41727740 0x55ac41718e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac4171eac0 0x55ac41733120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1491169922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4171cfa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1491169922 >> 172.21.15.31:0/1491169922 conn(0x55ac41718ab0 msgr2=0x55ac417376b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1491169922 shutdown_connections 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1491169922 wait complete. 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.533+0000 7f7cf95e3740 1 -- start start 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 Processor -- start 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4172e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac41727740 0x55ac41731e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac41739990 con 0x55ac4171d750 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ac417392c0 con 0x55ac41727740 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ac4173a060 con 0x55ac4171eac0 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 10 client.0 _ll_register_callbacks cb 0x7f7ce00036a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4172e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4172e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47854/0 (socket says 172.21.15.31:47854) 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1154236766 learned_addr learned my addr 172.21.15.31:0/1154236766 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7ceffff640 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 msgr2=0x55ac4172ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1154236766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac41727740 msgr2=0x55ac41731e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac41727740 0x55ac41731e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 -- 172.21.15.31:0/1154236766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac41732480 con 0x55ac4171d750 2024-04-25T00:29:55.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cef7fe640 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4172e930 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00037b0 tx=0x7f7ce0011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5102f7cdefe1c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7ccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/1154236766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0023070 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/1154236766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7ce00110d0 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.534+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce0019070 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/1154236766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ce001f410 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 10 client.15111 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac4179e760 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/1154236766 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ce0019070 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/1154236766 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7ce00047a0 con 0x55ac4171d750 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 10 client.15111 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 20 client.15111 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417a30a0 0x55ac417a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 4 client.15111 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:29:55.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 --> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] -- command(tid 0: {"prefix": "status"}) -- 0x55ac41688790 con 0x55ac417a30a0 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.535+0000 7f7ceffff640 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417a30a0 0x55ac417a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7ceffff640 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417a30a0 0x55ac417a1db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d78504515dd31a9 server_cookie=25dab5c15c3af6ea in_seq=0 out_seq=0 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7ccf7fe640 10 client.15111 ms_handle_connect on v2:172.21.15.169:6836/1684471092 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7ccf7fe640 1 -- 172.21.15.31:0/1154236766 <== mds.2 v2:172.21.15.169:6836/1684471092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ac41688790 con 0x55ac417a30a0 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7ccf7fe640 10 client.15111 handle_command_reply: tid=0 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 client.15111 shutdown 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 msgr2=0x55ac4172e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4172e930 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00037b0 tx=0x7f7ce0011c70 comp rx=0 tx=0).stop 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 shutdown_connections 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55ac417a30a0 0x55ac417a1db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac41727740 0x55ac41731e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7ccf7fe640 0 client.15111 ms_handle_reset on v2:172.21.15.169:6836/1684471092 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac4171eac0 0x55ac4172ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 --2- 172.21.15.31:0/1154236766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac4171d750 0x55ac4172e930 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 >> 172.21.15.31:0/1154236766 conn(0x55ac41718ab0 msgr2=0x55ac4171cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 shutdown_connections 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 1 -- 172.21.15.31:0/1154236766 wait complete. 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.536+0000 7f7cf95e3740 20 client.15111 trim_cache size 0 max 16384 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24499, 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:29:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:29:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 88.103657448000007, 2024-04-25T00:29:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:29:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T00:29:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:29:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 111.989796961 2024-04-25T00:29:55.544 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7cd80a2d30 msgr2=0x7f7cd80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7cd80a2d30 0x7f7cd80a51f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80bcb80 tx=0x7f7ce80dd040 comp rx=0 tx=0).stop 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 msgr2=0x7f7cf019dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf019dd70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009dc0 tx=0x7f7ce8003ba0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 shutdown_connections 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7cd80a2d30 0x7f7cd80a51f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cf0102250 0x7f7cf019dd70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cf0101890 0x7f7cf0196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 --2- 172.21.15.31:0/1601917513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf01006a0 0x7f7cf01967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 >> 172.21.15.31:0/1601917513 conn(0x7f7cf00fbc30 msgr2=0x7f7cf0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 shutdown_connections 2024-04-25T00:29:55.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.537+0000 7f7cf68d4640 1 -- 172.21.15.31:0/1601917513 wait complete. 2024-04-25T00:29:55.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:29:55.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 -- 172.21.15.31:0/3622126738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564102250 msgr2=0x7f7564108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 --2- 172.21.15.31:0/3622126738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564102250 0x7f7564108ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f755c009ca0 tx=0x7f755c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 -- 172.21.15.31:0/3622126738 shutdown_connections 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 --2- 172.21.15.31:0/3622126738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564102250 0x7f7564108ae0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 --2- 172.21.15.31:0/3622126738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7564101890 0x7f7564101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 --2- 172.21.15.31:0/3622126738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75641006a0 0x7f7564100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 -- 172.21.15.31:0/3622126738 >> 172.21.15.31:0/3622126738 conn(0x7f75640fbc30 msgr2=0x7f75640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 -- 172.21.15.31:0/3622126738 shutdown_connections 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 -- 172.21.15.31:0/3622126738 wait complete. 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.650+0000 7f7569c93640 1 Processor -- start 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 -- start start 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75641006a0 0x7f756406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 0x7f756419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7564102250 0x7f756419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f756410c1b0 con 0x7f7564101890 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f756410c030 con 0x7f7564102250 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7569c93640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f756410c330 con 0x7f75641006a0 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 0x7f756419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 0x7f756419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47876/0 (socket says 172.21.15.31:47876) 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 -- 172.21.15.31:0/3979055543 learned_addr learned my addr 172.21.15.31:0/3979055543 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f75637fe640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75641006a0 0x7f756406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 -- 172.21.15.31:0/3979055543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75641006a0 msgr2=0x7f756406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75641006a0 0x7f756406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7563fff640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7564102250 0x7f756419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 -- 172.21.15.31:0/3979055543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7564102250 msgr2=0x7f756419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7564102250 0x7f756419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 -- 172.21.15.31:0/3979055543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756419f5e0 con 0x7f7564101890 2024-04-25T00:29:55.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f75637fe640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75641006a0 0x7f756406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 0x7f756419c6b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f755800c890 tx=0x7f755800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.651+0000 7f7560ff9640 1 -- 172.21.15.31:0/3979055543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558004220 con 0x7f7564101890 2024-04-25T00:29:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.652+0000 7f7560ff9640 1 -- 172.21.15.31:0/3979055543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75580043c0 con 0x7f7564101890 2024-04-25T00:29:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.652+0000 7f7560ff9640 1 -- 172.21.15.31:0/3979055543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558004a00 con 0x7f7564101890 2024-04-25T00:29:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.652+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756419f870 con 0x7f7564101890 2024-04-25T00:29:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.652+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756419fdc0 con 0x7f7564101890 2024-04-25T00:29:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f7560ff9640 1 -- 172.21.15.31:0/3979055543 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7558004ba0 con 0x7f7564101890 2024-04-25T00:29:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 Processor -- start 2024-04-25T00:29:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 -- start start 2024-04-25T00:29:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d776c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d7771930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d7774430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621d76cff00 con 0x5621d776ce30 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621d7678750 con 0x5621d7771fa0 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f756c9a0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621d76cbec0 con 0x5621d776c440 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f75637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d7771930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f75637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d7771930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47884/0 (socket says 172.21.15.31:47884) 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.654+0000 7f75637fe640 1 -- 172.21.15.31:0/3623902133 learned_addr learned my addr 172.21.15.31:0/3623902133 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7560ff9640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f754c0a2b90 0x7f754c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d7774430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7563fff640 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d776c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f75637fe640 1 -- 172.21.15.31:0/3623902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 msgr2=0x5621d776c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7563fff640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f754c0a2b90 0x7f754c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f75637fe640 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d776c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f75637fe640 1 -- 172.21.15.31:0/3623902133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 msgr2=0x5621d7774430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f75637fe640 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d7774430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f75637fe640 1 -- 172.21.15.31:0/3623902133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d73e42d0 con 0x5621d776ce30 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d7774430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7560ff9640 1 -- 172.21.15.31:0/3979055543 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f75580cee20 con 0x7f7564101890 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f75637fe640 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d7771930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7554009c30 tx=0x7f755400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d97e9e18a9903cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7549ffb640 1 -- 172.21.15.31:0/3623902133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554016070 con 0x5621d776ce30 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7563fff640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f754c0a2b90 0x7f754c0a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f755c02ade0 tx=0x7f755c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7549ffb640 1 -- 172.21.15.31:0/3623902133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75540054d0 con 0x5621d776ce30 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.655+0000 7f7549ffb640 1 -- 172.21.15.31:0/3623902133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554011470 con 0x5621d776ce30 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 -- 172.21.15.31:0/3623902133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 msgr2=0x5621d7771930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d7771930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7554009c30 tx=0x7f755400e6c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 -- 172.21.15.31:0/3623902133 shutdown_connections 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d7774430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d7771930 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3623902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d776c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 -- 172.21.15.31:0/3623902133 >> 172.21.15.31:0/3623902133 conn(0x5621d7767ab0 msgr2=0x5621d7767eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 -- 172.21.15.31:0/3623902133 shutdown_connections 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 -- 172.21.15.31:0/3623902133 wait complete. 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 -- start start 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.656+0000 7f756c9a0740 1 Processor -- start 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 0x5621d7776890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776ce30 0x5621d7776dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d7771fa0 0x5621d777dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621d7774f50 con 0x5621d776c440 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621d768bcf0 con 0x5621d776ce30 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621d77750d0 con 0x5621d7771fa0 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 10 client.0 _ll_register_callbacks cb 0x5621d768b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 0x5621d7776890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 0x5621d7776890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47890/0 (socket says 172.21.15.31:47890) 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 -- 172.21.15.31:0/1019310246 learned_addr learned my addr 172.21.15.31:0/1019310246 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f7563fff640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d7771fa0 0x5621d777dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 -- 172.21.15.31:0/1019310246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d7771fa0 msgr2=0x5621d777dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d7771fa0 0x5621d777dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 -- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776ce30 msgr2=0x5621d7776dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776ce30 0x5621d7776dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 -- 172.21.15.31:0/1019310246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d74ae6b0 con 0x5621d776c440 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f7563fff640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d7771fa0 0x5621d777dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f75637fe640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 0x5621d7776890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7554000c00 tx=0x7f7554009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a88ee4e1d9debc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f7542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f7542ffd640 1 -- 172.21.15.31:0/1019310246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554028060 con 0x5621d776c440 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f7542ffd640 1 -- 172.21.15.31:0/1019310246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7554004f20 con 0x5621d776c440 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75540054d0 con 0x5621d776c440 2024-04-25T00:29:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f7542ffd640 1 -- 172.21.15.31:0/1019310246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7554016040 con 0x5621d776c440 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 10 client.15123 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.657+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621d7780df0 con 0x5621d776c440 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f7542ffd640 1 -- 172.21.15.31:0/1019310246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75540054d0 con 0x5621d776c440 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f7542ffd640 1 -- 172.21.15.31:0/1019310246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f755400f040 con 0x5621d776c440 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f756c9a0740 10 client.15123 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f756c9a0740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f756c9a0740 20 client.15123 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f756c9a0740 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d7783120 0x5621d7784590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f756c9a0740 4 client.15123 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 --> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621d76dd440 con 0x5621d7783120 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.658+0000 7f7562ffd640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d7783120 0x5621d7784590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.659+0000 7f7562ffd640 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d7783120 0x5621d7784590 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12b78981546b58c9 server_cookie=dc4ebecdfa6a0404 in_seq=0 out_seq=0 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.659+0000 7f7542ffd640 10 client.15123 ms_handle_connect on v2:172.21.15.40:6838/2724938586 2024-04-25T00:29:55.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.659+0000 7f7542ffd640 1 -- 172.21.15.31:0/1019310246 <== mds.1 v2:172.21.15.40:6838/2724938586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621d76dd440 con 0x5621d7783120 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.659+0000 7f7542ffd640 10 client.15123 handle_command_reply: tid=0 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 client.15123 shutdown 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 msgr2=0x5621d7776890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 0x5621d7776890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7554000c00 tx=0x7f7554009970 comp rx=0 tx=0).stop 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 shutdown_connections 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d7783120 0x5621d7784590 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d7771fa0 0x5621d777dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776ce30 0x5621d7776dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f7542ffd640 0 client.15123 ms_handle_reset on v2:172.21.15.40:6838/2724938586 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 --2- 172.21.15.31:0/1019310246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776c440 0x5621d7776890 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 >> 172.21.15.31:0/1019310246 conn(0x5621d7767ab0 msgr2=0x5621d7766ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 shutdown_connections 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 1 -- 172.21.15.31:0/1019310246 wait complete. 2024-04-25T00:29:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.660+0000 7f756c9a0740 20 client.15123 trim_cache size 0 max 16384 2024-04-25T00:29:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 Processor -- start 2024-04-25T00:29:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 -- start start 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776c440 0x5621d7767e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776ce30 0x5621d777dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 0x5621d7762690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621d77828d0 con 0x5621d7771fa0 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621d7782fa0 con 0x5621d776c440 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f756c9a0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621d7783670 con 0x5621d776ce30 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 0x5621d7762690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 0x5621d7762690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47906/0 (socket says 172.21.15.31:47906) 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 -- 172.21.15.31:0/2340592385 learned_addr learned my addr 172.21.15.31:0/2340592385 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 -- 172.21.15.31:0/2340592385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776ce30 msgr2=0x5621d777dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776ce30 0x5621d777dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 -- 172.21.15.31:0/2340592385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776c440 msgr2=0x5621d7767e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776c440 0x5621d7767e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.678+0000 7f7563fff640 1 -- 172.21.15.31:0/2340592385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d7768780 con 0x5621d7771fa0 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f7563fff640 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 0x5621d7762690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f755c009ca0 tx=0x7f755c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f85edfa857f5214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f75497fa640 1 -- 172.21.15.31:0/2340592385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c043070 con 0x5621d7771fa0 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f75497fa640 1 -- 172.21.15.31:0/2340592385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f755c036040 con 0x5621d7771fa0 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f75497fa640 1 -- 172.21.15.31:0/2340592385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c030a80 con 0x5621d7771fa0 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 -- 172.21.15.31:0/2340592385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 msgr2=0x5621d7762690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 0x5621d7762690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f755c009ca0 tx=0x7f755c03b000 comp rx=0 tx=0).stop 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 -- 172.21.15.31:0/2340592385 shutdown_connections 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d7771fa0 0x5621d7762690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776ce30 0x5621d777dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 --2- 172.21.15.31:0/2340592385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d776c440 0x5621d7767e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 -- 172.21.15.31:0/2340592385 >> 172.21.15.31:0/2340592385 conn(0x5621d7767ab0 msgr2=0x5621d776d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 -- 172.21.15.31:0/2340592385 shutdown_connections 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.679+0000 7f756c9a0740 1 -- 172.21.15.31:0/2340592385 wait complete. 2024-04-25T00:29:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 -- start start 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 Processor -- start 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d777bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d777c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d777ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621d7784410 con 0x5621d776ce30 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621d7783d40 con 0x5621d7771fa0 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621d7784ae0 con 0x5621d776c440 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f756c9a0740 10 client.0 _ll_register_callbacks cb 0x7f75540046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d777c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d777c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47908/0 (socket says 172.21.15.31:47908) 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 -- 172.21.15.31:0/3996652666 learned_addr learned my addr 172.21.15.31:0/3996652666 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f75637fe640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d777bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7563fff640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d777ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 -- 172.21.15.31:0/3996652666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 msgr2=0x5621d777bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d777bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 -- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 msgr2=0x5621d777ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d777ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.680+0000 7f7562ffd640 1 -- 172.21.15.31:0/3996652666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d7771cf0 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7562ffd640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d777c440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75580972f0 tx=0x7f75580971f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa14839ac95f775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7542ffd640 1 -- 172.21.15.31:0/3996652666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755801d040 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755c036040 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7542ffd640 1 -- 172.21.15.31:0/3996652666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f75580cc9c0 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7542ffd640 1 -- 172.21.15.31:0/3996652666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f755809f9c0 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 10 client.15129 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621d77e9320 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7542ffd640 1 -- 172.21.15.31:0/3996652666 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f755c036040 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f7542ffd640 1 -- 172.21.15.31:0/3996652666 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7554009d20 con 0x5621d776ce30 2024-04-25T00:29:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 10 client.15129 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 20 client.15129 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d77ede40 0x5621d77ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 4 client.15129 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.681+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 --> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] -- command(tid 0: {"prefix": "status"}) -- 0x5621d76d7790 con 0x5621d77ede40 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f75637fe640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d77ede40 0x5621d77ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f75637fe640 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d77ede40 0x5621d77ecb50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7102f9aeb0a2341b server_cookie=5641b60c0f28228b in_seq=0 out_seq=0 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f7542ffd640 10 client.15129 ms_handle_connect on v2:172.21.15.40:6838/2724938586 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f7542ffd640 1 -- 172.21.15.31:0/3996652666 <== mds.1 v2:172.21.15.40:6838/2724938586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5621d76d7790 con 0x5621d77ede40 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f7542ffd640 10 client.15129 handle_command_reply: tid=0 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 client.15129 shutdown 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 msgr2=0x5621d777c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d777c440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75580972f0 tx=0x7f75580971f0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 shutdown_connections 2024-04-25T00:29:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x5621d77ede40 0x5621d77ecb50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621d7771fa0 0x5621d777ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621d776ce30 0x5621d777c440 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 --2- 172.21.15.31:0/3996652666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621d776c440 0x5621d777bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f7542ffd640 0 client.15129 ms_handle_reset on v2:172.21.15.40:6838/2724938586 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 >> 172.21.15.31:0/3996652666 conn(0x5621d7767ab0 msgr2=0x5621d7762690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 shutdown_connections 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 1 -- 172.21.15.31:0/3996652666 wait complete. 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.682+0000 7f756c9a0740 20 client.15129 trim_cache size 0 max 16384 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24512, 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 89.258649133999995, 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 97.602292943999998 2024-04-25T00:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f754c0a2b90 msgr2=0x7f754c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f754c0a2b90 0x7f754c0a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f755c02ade0 tx=0x7f755c0095c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 msgr2=0x7f756419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 0x7f756419c6b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f755800c890 tx=0x7f755800cd50 comp rx=0 tx=0).stop 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 shutdown_connections 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7564102250 0x7f756419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f754c0a2b90 0x7f754c0a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7564101890 0x7f756419c6b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 --2- 172.21.15.31:0/3979055543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75641006a0 0x7f756406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 >> 172.21.15.31:0/3979055543 conn(0x7f75640fbc30 msgr2=0x7f7564104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 shutdown_connections 2024-04-25T00:29:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.683+0000 7f7569c93640 1 -- 172.21.15.31:0/3979055543 wait complete. 2024-04-25T00:29:55.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 -- 172.21.15.31:0/2961875346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 msgr2=0x7f20b8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/2961875346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8102e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009900 tx=0x7f20a002a410 comp rx=0 tx=0).stop 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 -- 172.21.15.31:0/2961875346 shutdown_connections 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/2961875346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b8103340 0x7f20b8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/2961875346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8102e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/2961875346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8101790 0x7f20b8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 -- 172.21.15.31:0/2961875346 >> 172.21.15.31:0/2961875346 conn(0x7f20b80fcd20 msgr2=0x7f20b80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 -- 172.21.15.31:0/2961875346 shutdown_connections 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.796+0000 7f20bdf9e640 1 -- 172.21.15.31:0/2961875346 wait complete. 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 Processor -- start 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 -- start start 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8101790 0x7f20b8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b8103340 0x7f20b819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810d2c0 con 0x7f20b8102980 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810d140 con 0x7f20b8101790 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20bdf9e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810d440 con 0x7f20b8103340 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47934/0 (socket says 172.21.15.31:47934) 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1678850503 learned_addr learned my addr 172.21.15.31:0/1678850503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.797+0000 7f20b77fe640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8101790 0x7f20b8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b7fff640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b8103340 0x7f20b819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1678850503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b8103340 msgr2=0x7f20b819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b8103340 0x7f20b819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1678850503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8101790 msgr2=0x7f20b8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8101790 0x7f20b8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1678850503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b819f580 con 0x7f20b8102980 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8197ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005940 tx=0x7f20a0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b4ff9640 1 -- 172.21.15.31:0/1678850503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a003f030 con 0x7f20b8102980 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b4ff9640 1 -- 172.21.15.31:0/1678850503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a0003cb0 con 0x7f20b8102980 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b819f810 con 0x7f20b8102980 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20b4ff9640 1 -- 172.21.15.31:0/1678850503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0035070 con 0x7f20b8102980 2024-04-25T00:29:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.798+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b819fd00 con 0x7f20b8102980 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b4ff9640 1 -- 172.21.15.31:0/1678850503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f20a0042030 con 0x7f20b8102980 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 Processor -- start 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 -- start start 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0440 0x55fd8b2d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2d5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b233f00 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b1dc750 con 0x55fd8b2d0e30 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20c0d85740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b22fec0 con 0x55fd8b2d0440 2024-04-25T00:29:55.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2d8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47944/0 (socket says 172.21.15.31:47944) 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1269732731 learned_addr learned my addr 172.21.15.31:0/1269732731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b7fff640 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0440 0x55fd8b2d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b77fe640 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2d5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b4ff9640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f20a40a2b90 0x7f20a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1269732731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0440 msgr2=0x55fd8b2d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0440 0x55fd8b2d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1269732731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 msgr2=0x55fd8b2d5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2d5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b6ffd640 1 -- 172.21.15.31:0/1269732731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8af482d0 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.801+0000 7f20b7fff640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f20a40a2b90 0x7f20a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20b4ff9640 1 -- 172.21.15.31:0/1678850503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f20a00efaf0 con 0x7f20b8102980 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2d8430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f20a002fc10 tx=0x7f20a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7210b52b871c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20adffb640 1 -- 172.21.15.31:0/1269732731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00f7040 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20b7fff640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f20a40a2b90 0x7f20a40a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55fd8b2d1690 tx=0x7f20b00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20adffb640 1 -- 172.21.15.31:0/1269732731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a00bca40 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20adffb640 1 -- 172.21.15.31:0/1269732731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00c06f0 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 -- 172.21.15.31:0/1269732731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 msgr2=0x55fd8b2d8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2d8430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f20a002fc10 tx=0x7f20a002a410 comp rx=0 tx=0).stop 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 -- 172.21.15.31:0/1269732731 shutdown_connections 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2d8430 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2d5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1269732731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0440 0x55fd8b2d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.802+0000 7f20c0d85740 1 -- 172.21.15.31:0/1269732731 >> 172.21.15.31:0/1269732731 conn(0x55fd8b2cbab0 msgr2=0x55fd8b2cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 -- 172.21.15.31:0/1269732731 shutdown_connections 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 -- 172.21.15.31:0/1269732731 wait complete. 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 -- start start 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 Processor -- start 2024-04-25T00:29:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2d8d10 con 0x55fd8b2d0440 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b1efcf0 con 0x55fd8b2d0e30 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2d8e90 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20c0d85740 10 client.0 _ll_register_callbacks cb 0x55fd8b1ef6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2da4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47946/0 (socket says 172.21.15.31:47946) 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20b77fe640 1 -- 172.21.15.31:0/786498127 learned_addr learned my addr 172.21.15.31:0/786498127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.803+0000 7f20b7fff640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b77fe640 1 -- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 msgr2=0x55fd8b2e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b77fe640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b77fe640 1 -- 172.21.15.31:0/786498127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 msgr2=0x55fd8b2daa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b77fe640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2daa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b77fe640 1 -- 172.21.15.31:0/786498127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8b0126b0 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b7fff640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20b77fe640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2da4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f20a8010b70 tx=0x7f20a800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121bfa830d47c718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f2096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f2096ffd640 1 -- 172.21.15.31:0/786498127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8003970 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f2096ffd640 1 -- 172.21.15.31:0/786498127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f20a8003b10 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a00bca40 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f2096ffd640 1 -- 172.21.15.31:0/786498127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20a8003d50 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20c0d85740 10 client.15141 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd8b2e4b10 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.804+0000 7f2096ffd640 1 -- 172.21.15.31:0/786498127 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a00bca40 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f2096ffd640 1 -- 172.21.15.31:0/786498127 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f20a800e7c0 con 0x55fd8b2d0440 2024-04-25T00:29:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20c0d85740 10 client.15141 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20c0d85740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20c0d85740 20 client.15141 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20c0d85740 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b2e71a0 0x55fd8b2e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20c0d85740 4 client.15141 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd8b241440 con 0x55fd8b2e71a0 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b2e71a0 0x55fd8b2e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b2e71a0 0x55fd8b2e8610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9235071565f66e3b server_cookie=a9298896a6e939b6 in_seq=0 out_seq=0 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.805+0000 7f2096ffd640 10 client.15141 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f2096ffd640 1 -- 172.21.15.31:0/786498127 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd8b241440 con 0x55fd8b2e71a0 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f2096ffd640 10 client.15141 handle_command_reply: tid=0 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 client.15141 shutdown 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 msgr2=0x55fd8b2da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2da4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f20a8010b70 tx=0x7f20a800f7f0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 shutdown_connections 2024-04-25T00:29:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b2e71a0 0x55fd8b2e8610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f2096ffd640 0 client.15141 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 --2- 172.21.15.31:0/786498127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2da4e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 >> 172.21.15.31:0/786498127 conn(0x55fd8b2cbab0 msgr2=0x55fd8b2caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 shutdown_connections 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 1 -- 172.21.15.31:0/786498127 wait complete. 2024-04-25T00:29:55.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.806+0000 7f20c0d85740 20 client.15141 trim_cache size 0 max 16384 2024-04-25T00:29:55.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 Processor -- start 2024-04-25T00:29:55.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 -- start start 2024-04-25T00:29:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2e6950 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2e7020 con 0x55fd8b2d0440 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20c0d85740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2e76f0 con 0x55fd8b2d0e30 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47948/0 (socket says 172.21.15.31:47948) 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 -- 172.21.15.31:0/2842291952 learned_addr learned my addr 172.21.15.31:0/2842291952 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 -- 172.21.15.31:0/2842291952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0e30 msgr2=0x55fd8b2e1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2e1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 -- 172.21.15.31:0/2842291952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0440 msgr2=0x55fd8b2ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 -- 172.21.15.31:0/2842291952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8b2d1210 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20b7fff640 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2ca790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f20b0011e50 tx=0x7f20b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8403bb4211e93a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20ad7fa640 1 -- 172.21.15.31:0/2842291952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0011e80 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20ad7fa640 1 -- 172.21.15.31:0/2842291952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20b001cb00 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.825+0000 7f20ad7fa640 1 -- 172.21.15.31:0/2842291952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b001ab70 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 -- 172.21.15.31:0/2842291952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 msgr2=0x55fd8b2ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2ca790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f20b0011e50 tx=0x7f20b0014000 comp rx=0 tx=0).stop 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 -- 172.21.15.31:0/2842291952 shutdown_connections 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2ca790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2e1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 --2- 172.21.15.31:0/2842291952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 -- 172.21.15.31:0/2842291952 >> 172.21.15.31:0/2842291952 conn(0x55fd8b2cbab0 msgr2=0x55fd8b2cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 -- 172.21.15.31:0/2842291952 shutdown_connections 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 -- 172.21.15.31:0/2842291952 wait complete. 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 -- start start 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.826+0000 7f20c0d85740 1 Processor -- start 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2e8490 con 0x55fd8b2d0440 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2e7dc0 con 0x55fd8b2d0e30 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fd8b2e8b60 con 0x55fd8b2d5fa0 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 10 client.0 _ll_register_callbacks cb 0x7f20a8010780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ddbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47954/0 (socket says 172.21.15.31:47954) 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 -- 172.21.15.31:0/1400372537 learned_addr learned my addr 172.21.15.31:0/1400372537 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b7fff640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 -- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 msgr2=0x55fd8b2e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 -- 172.21.15.31:0/1400372537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 msgr2=0x55fd8b2de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 -- 172.21.15.31:0/1400372537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8b2e11e0 con 0x55fd8b2d0440 2024-04-25T00:29:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20b77fe640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ddbb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f20a800ee20 tx=0x7f20a800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3bc8c69508aed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f2096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f2096ffd640 1 -- 172.21.15.31:0/1400372537 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a800e7d0 con 0x55fd8b2d0440 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b001cb00 con 0x55fd8b2d0440 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.827+0000 7f2096ffd640 1 -- 172.21.15.31:0/1400372537 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f20a800e970 con 0x55fd8b2d0440 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f2096ffd640 1 -- 172.21.15.31:0/1400372537 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20a80050e0 con 0x55fd8b2d0440 2024-04-25T00:29:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 10 client.15147 fetch_fsmap learned FSMap version 25 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd8b34d7e0 con 0x55fd8b2d0440 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f2096ffd640 1 -- 172.21.15.31:0/1400372537 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20b001cb00 con 0x55fd8b2d0440 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f2096ffd640 1 -- 172.21.15.31:0/1400372537 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f20a802d060 con 0x55fd8b2d0440 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 10 client.15147 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 10 client.15147 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 20 client.15147 populate_metadata read hostname 'smithi031' 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b352470 0x55fd8b34e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 4 client.15147 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "status"}) -- 0x55fd8b23b790 con 0x55fd8b352470 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.828+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b352470 0x55fd8b34e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20b6ffd640 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b352470 0x55fd8b34e8b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ae6f597d6ac97d server_cookie=623ffd3825148736 in_seq=0 out_seq=0 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f2096ffd640 10 client.15147 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f2096ffd640 1 -- 172.21.15.31:0/1400372537 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fd8b23b790 con 0x55fd8b352470 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f2096ffd640 10 client.15147 handle_command_reply: tid=0 2024-04-25T00:29:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 client.15147 shutdown 2024-04-25T00:29:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 msgr2=0x55fd8b2ddbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ddbb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f20a800ee20 tx=0x7f20a800eea0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 shutdown_connections 2024-04-25T00:29:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fd8b352470 0x55fd8b34e8b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd8b2d5fa0 0x55fd8b2e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd8b2d0e30 0x55fd8b2de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 --2- 172.21.15.31:0/1400372537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd8b2d0440 0x55fd8b2ddbb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f2096ffd640 0 client.15147 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:29:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 >> 172.21.15.31:0/1400372537 conn(0x55fd8b2cbab0 msgr2=0x55fd8b2ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 shutdown_connections 2024-04-25T00:29:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 1 -- 172.21.15.31:0/1400372537 wait complete. 2024-04-25T00:29:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.829+0000 7f20c0d85740 20 client.15147 trim_cache size 0 max 16384 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24517, 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 91.423382642999997, 2024-04-25T00:29:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:29:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T00:29:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:29:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 106.469566769 2024-04-25T00:29:55.845 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f20a40a2b90 msgr2=0x7f20a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f20a40a2b90 0x7f20a40a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55fd8b2d1690 tx=0x7f20b00046c0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 msgr2=0x7f20b8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8197ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005940 tx=0x7f20a0004430 comp rx=0 tx=0).stop 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 shutdown_connections 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b8103340 0x7f20b819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f20a40a2b90 0x7f20a40a5050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20b8102980 0x7f20b8197ea0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 --2- 172.21.15.31:0/1678850503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8101790 0x7f20b8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 >> 172.21.15.31:0/1678850503 conn(0x7f20b80fcd20 msgr2=0x7f20b8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 shutdown_connections 2024-04-25T00:29:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:55.830+0000 7f20bdf9e640 1 -- 172.21.15.31:0/1678850503 wait complete. 2024-04-25T00:29:55.847 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T00:29:55.847 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:29:55.864 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:29:55.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T00:29:56.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:55 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2827998683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:56.109 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:29:56.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:55 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2827998683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:56.146 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:55 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2827998683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:57.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:56 smithi169 ceph-mon[33112]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:57.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:56 smithi040 ceph-mon[32945]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:57.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:56 smithi031 ceph-mon[29167]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 -- 172.21.15.31:0/3488826002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c102390 msgr2=0x7f9e1c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 --2- 172.21.15.31:0/3488826002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c102390 0x7f9e1c1027f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9e100099c0 tx=0x7f9e10009cd0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 -- 172.21.15.31:0/3488826002 shutdown_connections 2024-04-25T00:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 --2- 172.21.15.31:0/3488826002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102d30 0x7f9e1c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 --2- 172.21.15.31:0/3488826002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c101190 0x7f9e1c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 --2- 172.21.15.31:0/3488826002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c102390 0x7f9e1c1027f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 -- 172.21.15.31:0/3488826002 >> 172.21.15.31:0/3488826002 conn(0x7f9e1c0fc730 msgr2=0x7f9e1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 -- 172.21.15.31:0/3488826002 shutdown_connections 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.063+0000 7f9e23e36700 1 -- 172.21.15.31:0/3488826002 wait complete. 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 Processor -- start 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 -- start start 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 0x7f9e1c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102390 0x7f9e1c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c102d30 0x7f9e1c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10cd20 con 0x7f9e1c101190 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10cba0 con 0x7f9e1c102390 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e23e36700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10cea0 con 0x7f9e1c102d30 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 0x7f9e1c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 0x7f9e1c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47960/0 (socket says 172.21.15.31:47960) 2024-04-25T00:29:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 -- 172.21.15.31:0/646818531 learned_addr learned my addr 172.21.15.31:0/646818531 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e213d1700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102390 0x7f9e1c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 -- 172.21.15.31:0/646818531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c102d30 msgr2=0x7f9e1c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e223d3700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c102d30 0x7f9e1c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c102d30 0x7f9e1c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 -- 172.21.15.31:0/646818531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102390 msgr2=0x7f9e1c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102390 0x7f9e1c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 -- 172.21.15.31:0/646818531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c1a0290 con 0x7f9e1c101190 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e213d1700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102390 0x7f9e1c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e223d3700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c102d30 0x7f9e1c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.064+0000 7f9e21bd2700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 0x7f9e1c198630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b9f0 tx=0x7f9e1400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.065+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14019070 con 0x7f9e1c101190 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.065+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c1a0580 con 0x7f9e1c101190 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.065+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e14014440 con 0x7f9e1c101190 2024-04-25T00:29:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.065+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14012580 con 0x7f9e1c101190 2024-04-25T00:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.065+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c1a0af0 con 0x7f9e1c101190 2024-04-25T00:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.066+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c068f20 con 0x7f9e1c101190 2024-04-25T00:29:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.067+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e1401c070 con 0x7f9e1c101190 2024-04-25T00:29:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.068+0000 7f9e1affd700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9e040a2c20 0x7f9e040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.068+0000 7f9e213d1700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9e040a2c20 0x7f9e040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.068+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f9e140ce4e0 con 0x7f9e1c101190 2024-04-25T00:29:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.069+0000 7f9e213d1700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9e040a2c20 0x7f9e040a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e10006010 tx=0x7f9e10005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.073+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e1409b430 con 0x7f9e1c101190 2024-04-25T00:29:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.301+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9e1c06a6b0 con 0x7f9e040a2c20 2024-04-25T00:29:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.303+0000 7f9e1affd700 1 -- 172.21.15.31:0/646818531 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9e1c06a6b0 con 0x7f9e040a2c20 2024-04-25T00:29:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-04-25T00:29:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-04-25T00:29:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-04-25T00:29:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9e040a2c20 msgr2=0x7f9e040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9e040a2c20 0x7f9e040a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e10006010 tx=0x7f9e10005c40 comp rx=0 tx=0).stop 2024-04-25T00:29:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 msgr2=0x7f9e1c198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 0x7f9e1c198630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b9f0 tx=0x7f9e1400bdb0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 shutdown_connections 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9e040a2c20 0x7f9e040a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c101190 0x7f9e1c198630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c102d30 0x7f9e1c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 --2- 172.21.15.31:0/646818531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1c102390 0x7f9e1c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 >> 172.21.15.31:0/646818531 conn(0x7f9e1c0fc730 msgr2=0x7f9e1c069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 shutdown_connections 2024-04-25T00:29:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:29:58.305+0000 7f9e23e36700 1 -- 172.21.15.31:0/646818531 wait complete. 2024-04-25T00:29:59.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:58 smithi031 ceph-mon[29167]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:59.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T00:29:59.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:58 smithi169 ceph-mon[33112]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:59.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:58 smithi040 ceph-mon[32945]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:29:59.392 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:00.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:29:59 smithi169 ceph-mon[33112]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:00.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:29:59 smithi040 ceph-mon[32945]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:00.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:29:59 smithi031 ceph-mon[29167]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:01.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:00 smithi169 ceph-mon[33112]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-25T00:30:01.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:00 smithi169 ceph-mon[33112]: overall HEALTH_OK 2024-04-25T00:30:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:00 smithi040 ceph-mon[32945]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-25T00:30:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:00 smithi040 ceph-mon[32945]: overall HEALTH_OK 2024-04-25T00:30:01.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:00 smithi031 ceph-mon[29167]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-25T00:30:01.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:00 smithi031 ceph-mon[29167]: overall HEALTH_OK 2024-04-25T00:30:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/1160527002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc103e30 msgr2=0x7fe9cc10a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/1160527002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc103e30 0x7fe9cc10a6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b0d0 tx=0x7fe9c400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/1160527002 shutdown_connections 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/1160527002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc103e30 0x7fe9cc10a6e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/1160527002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc102290 0x7fe9cc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/1160527002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9cc103490 0x7fe9cc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/1160527002 >> 172.21.15.31:0/1160527002 conn(0x7fe9cc0fd830 msgr2=0x7fe9cc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/1160527002 shutdown_connections 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.272+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/1160527002 wait complete. 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 Processor -- start 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 -- start start 2024-04-25T00:30:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 0x7fe9cc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc103490 0x7fe9cc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9cc103e30 0x7fe9cc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc10dd00 con 0x7fe9cc102290 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc10db80 con 0x7fe9cc103e30 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d2fe1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc10de80 con 0x7fe9cc103490 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d0d7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 0x7fe9cc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d0d7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 0x7fe9cc1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47982/0 (socket says 172.21.15.31:47982) 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d0d7d700 1 -- 172.21.15.31:0/4237003954 learned_addr learned my addr 172.21.15.31:0/4237003954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9cbfff700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc103490 0x7fe9cc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.273+0000 7fe9d157e700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9cc103e30 0x7fe9cc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d0d7d700 1 -- 172.21.15.31:0/4237003954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc103490 msgr2=0x7fe9cc198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d0d7d700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc103490 0x7fe9cc198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d0d7d700 1 -- 172.21.15.31:0/4237003954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9cc103e30 msgr2=0x7fe9cc19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d0d7d700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9cc103e30 0x7fe9cc19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d0d7d700 1 -- 172.21.15.31:0/4237003954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc19ff50 con 0x7fe9cc102290 2024-04-25T00:30:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d0d7d700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 0x7fe9cc1982f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000d9b0 tx=0x7fe9c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0017030 con 0x7fe9cc102290 2024-04-25T00:30:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc1a0240 con 0x7fe9cc102290 2024-04-25T00:30:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9c00044d0 con 0x7fe9cc102290 2024-04-25T00:30:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00038e0 con 0x7fe9cc102290 2024-04-25T00:30:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.274+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc1a0780 con 0x7fe9cc102290 2024-04-25T00:30:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.275+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9cc068f20 con 0x7fe9cc102290 2024-04-25T00:30:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.276+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9c001a070 con 0x7fe9cc102290 2024-04-25T00:30:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.277+0000 7fe9c9ffb700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe9bc0a2c20 0x7fe9bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.277+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe9c00ce910 con 0x7fe9cc102290 2024-04-25T00:30:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.277+0000 7fe9cbfff700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe9bc0a2c20 0x7fe9bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.278+0000 7fe9cbfff700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe9bc0a2c20 0x7fe9bc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800ac50 tx=0x7fe9b800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.282+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe9c009b860 con 0x7fe9cc102290 2024-04-25T00:30:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.501+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe9cc062670 con 0x7fe9bc0a2c20 2024-04-25T00:30:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.516+0000 7fe9c9ffb700 1 -- 172.21.15.31:0/4237003954 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fe9cc062670 con 0x7fe9bc0a2c20 2024-04-25T00:30:01.516 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (106s) 18s ago 106s 19.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef a44ed774e70d 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi040 running (103s) 37s ago 103s 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef f93e9953f001 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi169 running (118s) 37s ago 117s 20.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0b0fc29d034e 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (2m) 18s ago 2m 16.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1499cb49b90e 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi040 running (2m) 37s ago 2m 17.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef d18d84a04dc8 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi169 running (2m) 37s ago 2m 18.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef ecbb7508dd8a 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (109s) 18s ago 109s 16.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef d24fd09ac27c 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi040 running (115s) 37s ago 114s 16.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1eaedcfead6a 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi169 running (112s) 37s ago 112s 26.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 08f3ddf14c4c 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (2m) 18s ago 2m 20.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7ca3afb6f2a6 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi040 running (2m) 37s ago 2m 16.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 28e2f54d7edd 2024-04-25T00:30:01.517 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi169 running (2m) 37s ago 2m 17.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2200eac577be 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (9m) 18s ago 9m 514M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9b1f6788656b 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi040 *:8443,8765 running (7m) 37s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 889156ed9c11 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi169 *:8443,8765 running (7m) 37s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef cfba88e51db8 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (9m) 18s ago 9m 50.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 5218831c95b6 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi040 running (7m) 37s ago 7m 38.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef b28e66123773 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi169 running (8m) 37s ago 8m 40.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 3fe2bfd0089e 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 18s ago 7m 44.7M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef d640216608ef 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi040 running (6m) 37s ago 6m 55.1M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 1d1b75f5d807 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi169 running (6m) 37s ago 6m 47.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 9f5cd8848c24 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (6m) 18s ago 6m 57.8M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 09b72be50d9d 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (5m) 37s ago 5m 57.2M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef ff85eae39914 2024-04-25T00:30:01.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi169 running (5m) 37s ago 5m 55.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef e5491819dbec 2024-04-25T00:30:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (4m) 18s ago 4m 41.6M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef dd48adbfc11f 2024-04-25T00:30:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi040 running (4m) 37s ago 4m 54.2M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 3870ed79d0e1 2024-04-25T00:30:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi169 running (3m) 37s ago 3m 45.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 58cc51c66e50 2024-04-25T00:30:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 18s ago 3m 52.6M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 91a79568fae6 2024-04-25T00:30:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi040 running (3m) 37s ago 3m 46.4M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 3a374bfc2905 2024-04-25T00:30:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi169 running (2m) 37s ago 2m 41.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 1dcae73c7574 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe9bc0a2c20 msgr2=0x7fe9bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe9bc0a2c20 0x7fe9bc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800ac50 tx=0x7fe9b800a380 comp rx=0 tx=0).stop 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 msgr2=0x7fe9cc1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 0x7fe9cc1982f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000d9b0 tx=0x7fe9c000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 shutdown_connections 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe9bc0a2c20 0x7fe9bc0a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9cc102290 0x7fe9cc1982f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9cc103e30 0x7fe9cc19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 --2- 172.21.15.31:0/4237003954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc103490 0x7fe9cc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:01.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 >> 172.21.15.31:0/4237003954 conn(0x7fe9cc0fd830 msgr2=0x7fe9cc106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:01.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 shutdown_connections 2024-04-25T00:30:01.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:01.518+0000 7fe9d2fe1700 1 -- 172.21.15.31:0/4237003954 wait complete. 2024-04-25T00:30:02.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T00:30:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:02 smithi031 ceph-mon[29167]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:02 smithi031 ceph-mon[29167]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:03.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:02 smithi169 ceph-mon[33112]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:03.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:02 smithi169 ceph-mon[33112]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:02 smithi040 ceph-mon[32945]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:02 smithi040 ceph-mon[32945]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/4268868303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101470 msgr2=0x7fcd141018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/4268868303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101470 0x7fcd141018d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009990 tx=0x7fcd04009ca0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/4268868303 shutdown_connections 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/4268868303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14101e10 0x7fcd141096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/4268868303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd141027c0 0x7fcd14102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/4268868303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101470 0x7fcd141018d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/4268868303 >> 172.21.15.31:0/4268868303 conn(0x7fcd140fbf00 msgr2=0x7fcd140fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/4268868303 shutdown_connections 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/4268868303 wait complete. 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 Processor -- start 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 -- start start 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.320+0000 7fcd1c4cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd14101470 0x7fcd14198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 0x7fcd14198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd141027c0 0x7fcd1419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410cc40 con 0x7fcd14101e10 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410cac0 con 0x7fcd14101470 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410cdc0 con 0x7fcd141027c0 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 0x7fcd14198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 0x7fcd14198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44728/0 (socket says 172.21.15.31:44728) 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 -- 172.21.15.31:0/2496234780 learned_addr learned my addr 172.21.15.31:0/2496234780 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1a268700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd14101470 0x7fcd14198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1aa69700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd141027c0 0x7fcd1419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 -- 172.21.15.31:0/2496234780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd141027c0 msgr2=0x7fcd1419f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd141027c0 0x7fcd1419f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 -- 172.21.15.31:0/2496234780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd14101470 msgr2=0x7fcd14198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd14101470 0x7fcd14198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 -- 172.21.15.31:0/2496234780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd141a0020 con 0x7fcd14101e10 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd19a67700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 0x7fcd14198950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd04000c00 tx=0x7fcd040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04027070 con 0x7fcd14101e10 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd040040d0 con 0x7fcd14101e10 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd141a02b0 con 0x7fcd14101e10 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04022dc0 con 0x7fcd14101e10 2024-04-25T00:30:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.321+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd141a0700 con 0x7fcd14101e10 2024-04-25T00:30:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.322+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd14068f20 con 0x7fcd14101e10 2024-04-25T00:30:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.324+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcd0402a070 con 0x7fcd14101e10 2024-04-25T00:30:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.324+0000 7fcd0b7fe700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcd000a2c20 0x7fcd000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.325+0000 7fcd1a268700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcd000a2c20 0x7fcd000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.325+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcd040d05e0 con 0x7fcd14101e10 2024-04-25T00:30:04.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.325+0000 7fcd1a268700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcd000a2c20 0x7fcd000a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd1406a740 tx=0x7fcd1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.328+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcd04031070 con 0x7fcd14101e10 2024-04-25T00:30:04.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.544+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fcd140fe0f0 con 0x7fcd000a2c20 2024-04-25T00:30:04.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.551+0000 7fcd0b7fe700 1 -- 172.21.15.31:0/2496234780 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fcd140fe0f0 con 0x7fcd000a2c20 2024-04-25T00:30:04.551 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T00:30:04.551 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 40s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T00:30:04.551 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 40s ago 7m smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T00:30:04.551 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 40s ago 8m smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c;count:3 2024-04-25T00:30:04.552 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 40s ago - 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcd000a2c20 msgr2=0x7fcd000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcd000a2c20 0x7fcd000a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd1406a740 tx=0x7fcd1000b410 comp rx=0 tx=0).stop 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 msgr2=0x7fcd14198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 0x7fcd14198950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd04000c00 tx=0x7fcd040047b0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 shutdown_connections 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd141027c0 0x7fcd1419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd14101470 0x7fcd14198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcd000a2c20 0x7fcd000a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 --2- 172.21.15.31:0/2496234780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd14101e10 0x7fcd14198950 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 >> 172.21.15.31:0/2496234780 conn(0x7fcd140fbf00 msgr2=0x7fcd140fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 shutdown_connections 2024-04-25T00:30:04.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:04.553+0000 7fcd1c4cc700 1 -- 172.21.15.31:0/2496234780 wait complete. 2024-04-25T00:30:05.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:04 smithi169 ceph-mon[33112]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:05.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:04 smithi040 ceph-mon[32945]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:05.220 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:04 smithi031 ceph-mon[29167]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:05.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T00:30:05.501 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:05.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:05 smithi031 ceph-mon[29167]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:06.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:05 smithi169 ceph-mon[33112]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:06.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:05 smithi040 ceph-mon[32945]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:06 smithi031 ceph-mon[29167]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:07.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:06 smithi169 ceph-mon[33112]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:07.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:06 smithi040 ceph-mon[32945]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 -- 172.21.15.31:0/1754470882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35081023b0 msgr2=0x7f3508102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 --2- 172.21.15.31:0/1754470882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35081023b0 0x7f3508102810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009990 tx=0x7f34fc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 -- 172.21.15.31:0/1754470882 shutdown_connections 2024-04-25T00:30:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 --2- 172.21.15.31:0/1754470882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508102d50 0x7f3508109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 --2- 172.21.15.31:0/1754470882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35081011b0 0x7f35081015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 --2- 172.21.15.31:0/1754470882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35081023b0 0x7f3508102810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 -- 172.21.15.31:0/1754470882 >> 172.21.15.31:0/1754470882 conn(0x7f35080fc750 msgr2=0x7f35080feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 -- 172.21.15.31:0/1754470882 shutdown_connections 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 -- 172.21.15.31:0/1754470882 wait complete. 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 Processor -- start 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.288+0000 7f350e3dd700 1 -- start start 2024-04-25T00:30:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350e3dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35081011b0 0x7f35081983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350e3dd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35081023b0 0x7f3508198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350e3dd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 0x7f350819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350e3dd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f350810dd30 con 0x7f3508102d50 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350e3dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f350810dbb0 con 0x7f35081011b0 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350e3dd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f350810deb0 con 0x7f35081023b0 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 0x7f350819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 0x7f350819f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44754/0 (socket says 172.21.15.31:44754) 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 -- 172.21.15.31:0/3835785615 learned_addr learned my addr 172.21.15.31:0/3835785615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f35077fe700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35081023b0 0x7f3508198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f3507fff700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35081011b0 0x7f35081983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 -- 172.21.15.31:0/3835785615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35081023b0 msgr2=0x7f3508198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35081023b0 0x7f3508198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 -- 172.21.15.31:0/3835785615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35081011b0 msgr2=0x7f35081983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35081011b0 0x7f35081983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f350c97a700 1 -- 172.21.15.31:0/3835785615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081a0030 con 0x7f3508102d50 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.289+0000 7f3507fff700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35081011b0 0x7f35081983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.290+0000 7f350c97a700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 0x7f350819f990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f350000add0 tx=0x7f350000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.290+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500017020 con 0x7f3508102d50 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.290+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f350000ae00 con 0x7f3508102d50 2024-04-25T00:30:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.290+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500005350 con 0x7f3508102d50 2024-04-25T00:30:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.290+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35081a0320 con 0x7f3508102d50 2024-04-25T00:30:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.290+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a0810 con 0x7f3508102d50 2024-04-25T00:30:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.291+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3508068f20 con 0x7f3508102d50 2024-04-25T00:30:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.292+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f350000c940 con 0x7f3508102d50 2024-04-25T00:30:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.292+0000 7f35057fa700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f34f00a2c20 0x7f34f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.293+0000 7f3507fff700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f34f00a2c20 0x7f34f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.293+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f35000ce890 con 0x7f3508102d50 2024-04-25T00:30:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.293+0000 7f3507fff700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f34f00a2c20 0x7f34f00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f34f8005fd0 tx=0x7f34f8005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.297+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3500021020 con 0x7f3508102d50 2024-04-25T00:30:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.511+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3508062670 con 0x7f34f00a2c20 2024-04-25T00:30:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.512+0000 7f35057fa700 1 -- 172.21.15.31:0/3835785615 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3508062670 con 0x7f34f00a2c20 2024-04-25T00:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-04-25T00:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-04-25T00:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout:smithi040 172.21.15.40 2024-04-25T00:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout:smithi169 172.21.15.169 2024-04-25T00:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f34f00a2c20 msgr2=0x7f34f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f34f00a2c20 0x7f34f00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f34f8005fd0 tx=0x7f34f8005f00 comp rx=0 tx=0).stop 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 msgr2=0x7f350819f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 0x7f350819f990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f350000add0 tx=0x7f350000c330 comp rx=0 tx=0).stop 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 shutdown_connections 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f34f00a2c20 0x7f34f00a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508102d50 0x7f350819f990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35081011b0 0x7f35081983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 --2- 172.21.15.31:0/3835785615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35081023b0 0x7f3508198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 >> 172.21.15.31:0/3835785615 conn(0x7f35080fc750 msgr2=0x7f3508105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 shutdown_connections 2024-04-25T00:30:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:07.514+0000 7f350e3dd700 1 -- 172.21.15.31:0/3835785615 wait complete. 2024-04-25T00:30:08.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T00:30:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:09 smithi031 ceph-mon[29167]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:09 smithi031 ceph-mon[29167]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:09.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:09 smithi169 ceph-mon[33112]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:09.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:09 smithi169 ceph-mon[33112]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:09.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:09 smithi040 ceph-mon[32945]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:09.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:09 smithi040 ceph-mon[32945]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:10.332 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:10 smithi031 ceph-mon[29167]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T00:30:10.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 -- 172.21.15.31:0/1083084618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc101190 msgr2=0x7fddbc1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 --2- 172.21.15.31:0/1083084618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc101190 0x7fddbc1015b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b0d0 tx=0x7fddac00b3e0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 -- 172.21.15.31:0/1083084618 shutdown_connections 2024-04-25T00:30:10.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 --2- 172.21.15.31:0/1083084618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc101190 0x7fddbc1015b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 --2- 172.21.15.31:0/1083084618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddbc102d30 0x7fddbc1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 --2- 172.21.15.31:0/1083084618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc102390 0x7fddbc1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 -- 172.21.15.31:0/1083084618 >> 172.21.15.31:0/1083084618 conn(0x7fddbc0fc730 msgr2=0x7fddbc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 -- 172.21.15.31:0/1083084618 shutdown_connections 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.441+0000 7fddc30e5700 1 -- 172.21.15.31:0/1083084618 wait complete. 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 Processor -- start 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 -- start start 2024-04-25T00:30:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc101190 0x7fddbc196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddbc102390 0x7fddbc1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 0x7fddbc19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc10bbb0 con 0x7fddbc102d30 2024-04-25T00:30:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc10ba30 con 0x7fddbc101190 2024-04-25T00:30:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc30e5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc10bd30 con 0x7fddbc102390 2024-04-25T00:30:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc1682700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 0x7fddbc19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc1682700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 0x7fddbc19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44772/0 (socket says 172.21.15.31:44772) 2024-04-25T00:30:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc1682700 1 -- 172.21.15.31:0/4057671476 learned_addr learned my addr 172.21.15.31:0/4057671476 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddbbfff700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddbc102390 0x7fddbc1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.442+0000 7fddc0e81700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc101190 0x7fddbc196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc1682700 1 -- 172.21.15.31:0/4057671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddbc102390 msgr2=0x7fddbc1967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc1682700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddbc102390 0x7fddbc1967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc1682700 1 -- 172.21.15.31:0/4057671476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc101190 msgr2=0x7fddbc196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc1682700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc101190 0x7fddbc196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc1682700 1 -- 172.21.15.31:0/4057671476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc19dec0 con 0x7fddbc102d30 2024-04-25T00:30:10.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc0e81700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc101190 0x7fddbc196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc1682700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 0x7fddbc19d820 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e970 tx=0x7fddb400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb400cbd0 con 0x7fddbc102d30 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddb400cd70 con 0x7fddbc102d30 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc19e1b0 con 0x7fddbc102d30 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4005620 con 0x7fddbc102d30 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.443+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc19e740 con 0x7fddbc102d30 2024-04-25T00:30:10.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.444+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc068f20 con 0x7fddbc102d30 2024-04-25T00:30:10.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.445+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fddb4004750 con 0x7fddbc102d30 2024-04-25T00:30:10.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.446+0000 7fddb9ffb700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdda80a4eb0 0x7fdda80a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.446+0000 7fddc0e81700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdda80a4eb0 0x7fdda80a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.446+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fddb40cf1f0 con 0x7fddbc102d30 2024-04-25T00:30:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.447+0000 7fddc0e81700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdda80a4eb0 0x7fdda80a7370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fddac009be0 tx=0x7fddac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.451+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fddb409c140 con 0x7fddbc102d30 2024-04-25T00:30:10.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:10 smithi169 ceph-mon[33112]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T00:30:10.644 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:10 smithi040 ceph-mon[32945]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T00:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.664+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fddbc062670 con 0x7fdda80a4eb0 2024-04-25T00:30:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.666+0000 7fddb9ffb700 1 -- 172.21.15.31:0/4057671476 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fddbc062670 con 0x7fdda80a4eb0 2024-04-25T00:30:10.667 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T00:30:10.667 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:30:10.667 INFO:teuthology.orchestra.run.smithi031.stdout:smithi040 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:30:10.667 INFO:teuthology.orchestra.run.smithi031.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T00:30:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdda80a4eb0 msgr2=0x7fdda80a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdda80a4eb0 0x7fdda80a7370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fddac009be0 tx=0x7fddac00b040 comp rx=0 tx=0).stop 2024-04-25T00:30:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 msgr2=0x7fddbc19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 0x7fddbc19d820 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e970 tx=0x7fddb400ec80 comp rx=0 tx=0).stop 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 shutdown_connections 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdda80a4eb0 0x7fdda80a7370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddbc102d30 0x7fddbc19d820 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddbc101190 0x7fddbc196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 --2- 172.21.15.31:0/4057671476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddbc102390 0x7fddbc1967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.669+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 >> 172.21.15.31:0/4057671476 conn(0x7fddbc0fc730 msgr2=0x7fddbc105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.670+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 shutdown_connections 2024-04-25T00:30:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:10.670+0000 7fddc30e5700 1 -- 172.21.15.31:0/4057671476 wait complete. 2024-04-25T00:30:11.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:11 smithi031 ceph-mon[29167]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:11.497 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:30:11.506 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:30:11.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T00:30:11.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:11 smithi169 ceph-mon[33112]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:11.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:11 smithi040 ceph-mon[32945]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:12.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:12 smithi031 ceph-mon[29167]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:12.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:12 smithi169 ceph-mon[33112]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:12.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:12 smithi040 ceph-mon[32945]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3866804832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 msgr2=0x7f0fa41026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3866804832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa41026d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0f940099c0 tx=0x7f0f94009cd0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3866804832 shutdown_connections 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3866804832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa41026d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3866804832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa4103e50 0x7f0fa410a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3866804832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 0x7f0fa4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3866804832 >> 172.21.15.31:0/3866804832 conn(0x7f0fa40fd850 msgr2=0x7f0fa40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3866804832 shutdown_connections 2024-04-25T00:30:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.531+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3866804832 wait complete. 2024-04-25T00:30:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 Processor -- start 2024-04-25T00:30:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 -- start start 2024-04-25T00:30:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 0x7f0fa4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa4103e50 0x7f0fa419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa410de20 con 0x7f0fa41022b0 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa410dca0 con 0x7f0fa4103e50 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa9c9d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa410dfa0 con 0x7f0fa41034b0 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.532+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55764/0 (socket says 172.21.15.31:55764) 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 -- 172.21.15.31:0/3563607551 learned_addr learned my addr 172.21.15.31:0/3563607551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa2ffd700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 0x7f0fa4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa3fff700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa4103e50 0x7f0fa419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 -- 172.21.15.31:0/3563607551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 msgr2=0x7f0fa4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 0x7f0fa4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 -- 172.21.15.31:0/3563607551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa4103e50 msgr2=0x7f0fa419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa4103e50 0x7f0fa419fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 -- 172.21.15.31:0/3563607551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa41a01b0 con 0x7f0fa41022b0 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa2ffd700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 0x7f0fa4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa37fe700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa4198550 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0f94003680 tx=0x7f0f94005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94027070 con 0x7f0fa41022b0 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f94022470 con 0x7f0fa41022b0 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa41a0440 con 0x7f0fa41022b0 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9400f6c0 con 0x7f0fa41022b0 2024-04-25T00:30:13.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.533+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41a09f0 con 0x7f0fa41022b0 2024-04-25T00:30:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.534+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa4068f20 con 0x7f0fa41022b0 2024-04-25T00:30:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.536+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f9402a070 con 0x7f0fa41022b0 2024-04-25T00:30:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.536+0000 7f0fa0ff9700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0f8c0a2ca0 0x7f0f8c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.536+0000 7f0fa2ffd700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0f8c0a2ca0 0x7f0f8c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.536+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f0f940cfea0 con 0x7f0fa41022b0 2024-04-25T00:30:13.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.537+0000 7f0fa2ffd700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0f8c0a2ca0 0x7f0f8c0a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007900 tx=0x7f0f98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:13.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.541+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0f94013070 con 0x7f0fa41022b0 2024-04-25T00:30:13.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.793+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0fa4062e40 con 0x7f0fa41022b0 2024-04-25T00:30:13.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.794+0000 7f0fa0ff9700 1 -- 172.21.15.31:0/3563607551 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3086 (secure 0 0 0) 0x7f0f9409cfa0 con 0x7f0fa41022b0 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:e25 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 25 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-25T00:28:23.753095+0000 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-25T00:28:30.181657+0000 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-25T00:30:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 5 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1,2,3,4 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24517,1=24512,2=24499,3=24434,4=24424} 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-25T00:30:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24517 members: 24512,24517,24499,24434,24424 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{0:24517} state up:active seq 5 addr [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{0:24481} state up:standby-replay seq 1 addr [v2:172.21.15.169:6834/3258138877,v1:172.21.15.169:6835/3258138877] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{1:24512} state up:active seq 4 addr [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.40:6836/3765244618,v1:172.21.15.40:6837/3765244618] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{2:24499} state up:active seq 7 addr [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{2:24455} state up:standby-replay seq 1 addr [v2:172.21.15.40:6834/653343360,v1:172.21.15.40:6835/653343360] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{3:24434} state up:active seq 11 addr [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.797 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{3:24416} state up:standby-replay seq 1 addr [v2:172.21.15.40:6832/3504624607,v1:172.21.15.40:6833/3504624607] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{4:24424} state up:active seq 12 addr [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{4:14643} state up:standby-replay seq 1 addr [v2:172.21.15.31:6840/2045809592,v1:172.21.15.31:6841/2045809592] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.31:6836/951372407,v1:172.21.15.31:6837/951372407] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.31:6838/2151696509,v1:172.21.15.31:6839/2151696509] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:30:13.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0f8c0a2ca0 msgr2=0x7f0f8c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:13.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0f8c0a2ca0 0x7f0f8c0a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007900 tx=0x7f0f98008040 comp rx=0 tx=0).stop 2024-04-25T00:30:13.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 msgr2=0x7f0fa4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:13.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa4198550 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0f94003680 tx=0x7f0f94005e80 comp rx=0 tx=0).stop 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 shutdown_connections 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f0f8c0a2ca0 0x7f0f8c0a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fa41022b0 0x7f0fa4198550 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa4103e50 0x7f0fa419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 --2- 172.21.15.31:0/3563607551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fa41034b0 0x7f0fa4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 >> 172.21.15.31:0/3563607551 conn(0x7f0fa40fd850 msgr2=0x7f0fa41068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 shutdown_connections 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:13.796+0000 7f0fa9c9d700 1 -- 172.21.15.31:0/3563607551 wait complete. 2024-04-25T00:30:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:30:14.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:14 smithi031 ceph-mon[29167]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:14.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:14 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T00:30:14.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:14 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3563607551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:30:14.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T00:30:14.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:14 smithi169 ceph-mon[33112]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:14.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:14 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T00:30:14.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:14 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3563607551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:30:14.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:14 smithi040 ceph-mon[32945]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:14.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:14 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T00:30:14.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:14 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3563607551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:30:14.797 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:15.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:15 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T00:30:15.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:15 smithi031 ceph-mon[29167]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:30:15.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:15 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T00:30:15.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:15 smithi169 ceph-mon[33112]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:30:15.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:15 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T00:30:15.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:15 smithi040 ceph-mon[32945]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:30:16.668 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:16 smithi031 ceph-mon[29167]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:30:16.668 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:16 smithi031 ceph-mon[29167]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:30:16.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 -- 172.21.15.31:0/2571204149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 msgr2=0x7fa4bc10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/2571204149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc10a3f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400b0a0 tx=0x7fa4b400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 -- 172.21.15.31:0/2571204149 shutdown_connections 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/2571204149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc10a3f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/2571204149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4bc102d80 0x7fa4bc105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/2571204149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc1056b0 0x7fa4bc107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 -- 172.21.15.31:0/2571204149 >> 172.21.15.31:0/2571204149 conn(0x7fa4bc0fc730 msgr2=0x7fa4bc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 -- 172.21.15.31:0/2571204149 shutdown_connections 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.683+0000 7fa4c0d34700 1 -- 172.21.15.31:0/2571204149 wait complete. 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 Processor -- start 2024-04-25T00:30:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 -- start start 2024-04-25T00:30:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc102d80 0x7fa4bc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4bc1056b0 0x7fa4bc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc10dd00 con 0x7fa4bc107fe0 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc10db80 con 0x7fa4bc1056b0 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4c0d34700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc10de80 con 0x7fa4bc102d80 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4bad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4bad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc19f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55790/0 (socket says 172.21.15.31:55790) 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.684+0000 7fa4bad9d700 1 -- 172.21.15.31:0/654360110 learned_addr learned my addr 172.21.15.31:0/654360110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4b9d9b700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4bc1056b0 0x7fa4bc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4ba59c700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc102d80 0x7fa4bc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4bad9d700 1 -- 172.21.15.31:0/654360110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc102d80 msgr2=0x7fa4bc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4bad9d700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc102d80 0x7fa4bc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4bad9d700 1 -- 172.21.15.31:0/654360110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4bc1056b0 msgr2=0x7fa4bc1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4bad9d700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4bc1056b0 0x7fa4bc1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4bad9d700 1 -- 172.21.15.31:0/654360110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc19fff0 con 0x7fa4bc107fe0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4ba59c700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc102d80 0x7fa4bc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4bad9d700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc19f950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006070 tx=0x7fa4b4004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4021070 con 0x7fa4bc107fe0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b4007c80 con 0x7fa4bc107fe0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4bc1a0280 con 0x7fa4bc107fe0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b401d450 con 0x7fa4bc107fe0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.685+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4bc1a06e0 con 0x7fa4bc107fe0 2024-04-25T00:30:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.686+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4bc068f20 con 0x7fa4bc107fe0 2024-04-25T00:30:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.687+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4b4003a00 con 0x7fa4bc107fe0 2024-04-25T00:30:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.688+0000 7fa4ab7fe700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.688+0000 7fa4ba59c700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.688+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa4b40d0620 con 0x7fa4bc107fe0 2024-04-25T00:30:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.688+0000 7fa4ba59c700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005fd0 tx=0x7fa4b0005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.692+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4b409d580 con 0x7fa4bc107fe0 2024-04-25T00:30:16.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:16 smithi169 ceph-mon[33112]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:30:16.891 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:16 smithi169 ceph-mon[33112]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:30:16.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:16 smithi040 ceph-mon[32945]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:30:16.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:16 smithi040 ceph-mon[32945]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:30:16.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.899+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa4bc062e40 con 0x7fa4bc107fe0 2024-04-25T00:30:16.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.900+0000 7fa4ab7fe700 1 -- 172.21.15.31:0/654360110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5337 (secure 0 0 0) 0x7fa4b409ce80 con 0x7fa4bc107fe0 2024-04-25T00:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 91 2024-04-25T00:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-25T00:20:13.144267+0000 2024-04-25T00:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-25T00:30:15.394985+0000 2024-04-25T00:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-04-25T00:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/2938713990,v1:172.21.15.31:6803/2938713990] [v2:172.21.15.31:6804/2938713990,v1:172.21.15.31:6805/2938713990] exists,up a5605681-c8ce-4528-a557-32de873bf2a3 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6800/2453389189,v1:172.21.15.40:6801/2453389189] [v2:172.21.15.40:6802/2453389189,v1:172.21.15.40:6803/2453389189] exists,up 32f950d6-4574-449b-8273-82833bc7774d 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/1729358073,v1:172.21.15.169:6801/1729358073] [v2:172.21.15.169:6802/1729358073,v1:172.21.15.169:6803/1729358073] exists,up 1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 26 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/2474264832,v1:172.21.15.31:6811/2474264832] [v2:172.21.15.31:6812/2474264832,v1:172.21.15.31:6813/2474264832] exists,up 938ed94c-292f-4151-ad75-7a901afb1717 2024-04-25T00:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 34 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6808/3167270680,v1:172.21.15.40:6809/3167270680] [v2:172.21.15.40:6810/3167270680,v1:172.21.15.40:6811/3167270680] exists,up d9d02417-522a-472c-bc86-eb4a65662c6c 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/4050548275,v1:172.21.15.169:6809/4050548275] [v2:172.21.15.169:6810/4050548275,v1:172.21.15.169:6811/4050548275] exists,up c51c2807-c67a-46ef-8add-0ee8009cdd3e 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/3697284546,v1:172.21.15.31:6819/3697284546] [v2:172.21.15.31:6820/3697284546,v1:172.21.15.31:6821/3697284546] exists,up c4034099-a0bd-4ef2-95a1-3b00255b587e 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 53 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6816/4053415057,v1:172.21.15.40:6817/4053415057] [v2:172.21.15.40:6818/4053415057,v1:172.21.15.40:6819/4053415057] exists,up cc3fde10-5c71-4d9b-a4b0-b184780d5286 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 61 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/3220692863,v1:172.21.15.169:6817/3220692863] [v2:172.21.15.169:6818/3220692863,v1:172.21.15.169:6819/3220692863] exists,up ddad000a-a81c-4a3d-b5ef-ca98e6a57bab 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 69 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/1961015503,v1:172.21.15.31:6827/1961015503] [v2:172.21.15.31:6828/1961015503,v1:172.21.15.31:6829/1961015503] exists,up d854d1bd-461f-4def-b383-20908701640b 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6824/3974437906,v1:172.21.15.40:6825/3974437906] [v2:172.21.15.40:6826/3974437906,v1:172.21.15.40:6827/3974437906] exists,up 413c2a5c-f9a1-4a20-838d-624a8afae359 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/3987371467,v1:172.21.15.169:6825/3987371467] [v2:172.21.15.169:6826/3987371467,v1:172.21.15.169:6827/3987371467] exists,up 0d81d667-30bf-47da-824f-51eb05444477 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.a [2,11,8,6] 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.10 [2,4] 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1a [8,1] 2024-04-25T00:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1f [7,11] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.0 [4,6] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1b [3,8] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.28 [5,8] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.34 [4,9] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.38 [3,9] 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3395718020 expires 2024-04-26T00:21:13.218737+0000 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/3443312222 expires 2024-04-26T00:21:13.218737+0000 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/3443312222 expires 2024-04-26T00:21:13.218737+0000 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3607070745 expires 2024-04-26T00:20:45.382327+0000 2024-04-25T00:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3134587413 expires 2024-04-26T00:20:45.382327+0000 2024-04-25T00:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3135101963 expires 2024-04-26T00:21:13.218737+0000 2024-04-25T00:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/1911777180 expires 2024-04-26T00:20:45.382327+0000 2024-04-25T00:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3278889828 expires 2024-04-26T00:21:13.218737+0000 2024-04-25T00:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/1911777180 expires 2024-04-26T00:20:45.382327+0000 2024-04-25T00:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/4126189511 expires 2024-04-26T00:20:45.382327+0000 2024-04-25T00:30:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa4a40a2c20 msgr2=0x7fa4a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005fd0 tx=0x7fa4b0005f60 comp rx=0 tx=0).stop 2024-04-25T00:30:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 msgr2=0x7fa4bc19f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc19f950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006070 tx=0x7fa4b4004e40 comp rx=0 tx=0).stop 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 shutdown_connections 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4bc107fe0 0x7fa4bc19f950 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4bc102d80 0x7fa4bc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 --2- 172.21.15.31:0/654360110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4bc1056b0 0x7fa4bc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 >> 172.21.15.31:0/654360110 conn(0x7fa4bc0fc730 msgr2=0x7fa4bc1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 shutdown_connections 2024-04-25T00:30:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:16.903+0000 7fa4c0d34700 1 -- 172.21.15.31:0/654360110 wait complete. 2024-04-25T00:30:17.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:17 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/654360110' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:30:17.829 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T00:30:18.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:30:18.052 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:17 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/654360110' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:30:18.052 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:17 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/654360110' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:30:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 -- 172.21.15.31:0/58929439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49441006a0 msgr2=0x7f4944100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 --2- 172.21.15.31:0/58929439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49441006a0 0x7f4944100ac0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4940009900 tx=0x7f494002a410 comp rx=0 tx=0).stop 2024-04-25T00:30:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 -- 172.21.15.31:0/58929439 shutdown_connections 2024-04-25T00:30:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 --2- 172.21.15.31:0/58929439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4944102250 0x7f4944108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 --2- 172.21.15.31:0/58929439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944101890 0x7f4944101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 --2- 172.21.15.31:0/58929439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49441006a0 0x7f4944100ac0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 -- 172.21.15.31:0/58929439 >> 172.21.15.31:0/58929439 conn(0x7f49440fbc30 msgr2=0x7f49440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 -- 172.21.15.31:0/58929439 shutdown_connections 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.168+0000 7f494c529640 1 -- 172.21.15.31:0/58929439 wait complete. 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 Processor -- start 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 -- start start 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49441006a0 0x7f4944197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 0x7f4944197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4944102250 0x7f494419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f494410d2f0 con 0x7f4944101890 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f494410d170 con 0x7f4944102250 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f494c529640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f494410d470 con 0x7f49441006a0 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.169+0000 7f4949ac4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 0x7f4944197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 0x7f4944197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55822/0 (socket says 172.21.15.31:55822) 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 -- 172.21.15.31:0/2643889350 learned_addr learned my addr 172.21.15.31:0/2643889350 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f494aac6640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4944102250 0x7f494419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f494a2c5640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49441006a0 0x7f4944197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 -- 172.21.15.31:0/2643889350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49441006a0 msgr2=0x7f4944197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49441006a0 0x7f4944197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 -- 172.21.15.31:0/2643889350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4944102250 msgr2=0x7f494419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4944102250 0x7f494419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 -- 172.21.15.31:0/2643889350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494419f650 con 0x7f4944101890 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f494a2c5640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49441006a0 0x7f4944197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f4949ac4640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 0x7f4944197f70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f493400e9f0 tx=0x7f493400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400cd80 con 0x7f4944101890 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49340054d0 con 0x7f4944101890 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494419f940 con 0x7f4944101890 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934010490 con 0x7f4944101890 2024-04-25T00:30:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.170+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494419fe40 con 0x7f4944101890 2024-04-25T00:30:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.171+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4944107b50 con 0x7f4944101890 2024-04-25T00:30:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.172+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4934005020 con 0x7f4944101890 2024-04-25T00:30:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.173+0000 7f493b7fe640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49300a2b90 0x7f49300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.173+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f49340cf5b0 con 0x7f4944101890 2024-04-25T00:30:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.173+0000 7f494a2c5640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49300a2b90 0x7f49300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.173+0000 7f494a2c5640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49300a2b90 0x7f49300a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4940000c00 tx=0x7f4940032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.177+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f493409c480 con 0x7f4944101890 2024-04-25T00:30:18.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.364+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4944062340 con 0x7f4944101890 2024-04-25T00:30:18.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.365+0000 7f493b7fe640 1 -- 172.21.15.31:0/2643889350 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7f493409bd80 con 0x7f4944101890 2024-04-25T00:30:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:18.366 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49300a2b90 msgr2=0x7f49300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49300a2b90 0x7f49300a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4940000c00 tx=0x7f4940032c40 comp rx=0 tx=0).stop 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 msgr2=0x7f4944197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 0x7f4944197f70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f493400e9f0 tx=0x7f493400edf0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 shutdown_connections 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4944102250 0x7f494419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f49300a2b90 0x7f49300a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4944101890 0x7f4944197f70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 --2- 172.21.15.31:0/2643889350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49441006a0 0x7f4944197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 >> 172.21.15.31:0/2643889350 conn(0x7f49440fbc30 msgr2=0x7f4944104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 shutdown_connections 2024-04-25T00:30:18.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.367+0000 7f494c529640 1 -- 172.21.15.31:0/2643889350 wait complete. 2024-04-25T00:30:18.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:30:18.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:30:18.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 -- 172.21.15.31:0/2335958630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 msgr2=0x7f8240108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 --2- 172.21.15.31:0/2335958630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f8240108ae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ca0 tx=0x7f823802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 -- 172.21.15.31:0/2335958630 shutdown_connections 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 --2- 172.21.15.31:0/2335958630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f8240108ae0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 --2- 172.21.15.31:0/2335958630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8240101890 0x7f8240101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 --2- 172.21.15.31:0/2335958630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82401006a0 0x7f8240100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 -- 172.21.15.31:0/2335958630 >> 172.21.15.31:0/2335958630 conn(0x7f82400fbc30 msgr2=0x7f82400fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 -- 172.21.15.31:0/2335958630 shutdown_connections 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 -- 172.21.15.31:0/2335958630 wait complete. 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 Processor -- start 2024-04-25T00:30:18.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.485+0000 7f82466b3640 1 -- start start 2024-04-25T00:30:18.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82401006a0 0x7f82401979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8240101890 0x7f8240197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f824019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f824010d2f0 con 0x7f8240102250 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f824010d170 con 0x7f82401006a0 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f824010d470 con 0x7f8240101890 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f824019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f824019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55828/0 (socket says 172.21.15.31:55828) 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 -- 172.21.15.31:0/4126043179 learned_addr learned my addr 172.21.15.31:0/4126043179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f823f7fe640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8240101890 0x7f8240197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f823ffff640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82401006a0 0x7f82401979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 -- 172.21.15.31:0/4126043179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8240101890 msgr2=0x7f8240197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8240101890 0x7f8240197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 -- 172.21.15.31:0/4126043179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82401006a0 msgr2=0x7f82401979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82401006a0 0x7f82401979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 -- 172.21.15.31:0/4126043179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824019f5c0 con 0x7f8240102250 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f823ffff640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82401006a0 0x7f82401979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f8244c50640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f824019ef50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8238031970 tx=0x7f8238003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238004040 con 0x7f8240102250 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8238004690 con 0x7f8240102250 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238030c60 con 0x7f8240102250 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.486+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824019f850 con 0x7f8240102250 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.487+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824019fdf0 con 0x7f8240102250 2024-04-25T00:30:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.487+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82401079c0 con 0x7f8240102250 2024-04-25T00:30:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.489+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82380041e0 con 0x7f8240102250 2024-04-25T00:30:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.489+0000 7f823d7fa640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f822c0a2c10 0x7f822c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.490+0000 7f823ffff640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f822c0a2c10 0x7f822c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.490+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f82380f06a0 con 0x7f8240102250 2024-04-25T00:30:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.490+0000 7f823ffff640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f822c0a2c10 0x7f822c0a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f82401016f0 tx=0x7f8228005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.493+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82380bd5d0 con 0x7f8240102250 2024-04-25T00:30:18.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.682+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f824002d300 con 0x7f8240102250 2024-04-25T00:30:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.682+0000 7f823d7fa640 1 -- 172.21.15.31:0/4126043179 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f82380bced0 con 0x7f8240102250 2024-04-25T00:30:18.683 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f822c0a2c10 msgr2=0x7f822c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f822c0a2c10 0x7f822c0a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f82401016f0 tx=0x7f8228005f90 comp rx=0 tx=0).stop 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 msgr2=0x7f824019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f824019ef50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8238031970 tx=0x7f8238003ba0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 shutdown_connections 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f822c0a2c10 0x7f822c0a50d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8240102250 0x7f824019ef50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8240101890 0x7f8240197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 --2- 172.21.15.31:0/4126043179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82401006a0 0x7f82401979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.684+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 >> 172.21.15.31:0/4126043179 conn(0x7f82400fbc30 msgr2=0x7f8240104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.685+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 shutdown_connections 2024-04-25T00:30:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.685+0000 7f82466b3640 1 -- 172.21.15.31:0/4126043179 wait complete. 2024-04-25T00:30:18.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:18 smithi031 ceph-mon[29167]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-04-25T00:30:18.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:18 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2643889350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:18.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:30:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 -- 172.21.15.31:0/2960199299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe354102250 msgr2=0x7fe354108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/2960199299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe354102250 0x7fe354108ae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009ca0 tx=0x7fe34c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 -- 172.21.15.31:0/2960199299 shutdown_connections 2024-04-25T00:30:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/2960199299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe354102250 0x7fe354108ae0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/2960199299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe354101890 0x7fe354101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/2960199299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3541006a0 0x7fe354100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 -- 172.21.15.31:0/2960199299 >> 172.21.15.31:0/2960199299 conn(0x7fe3540fbc30 msgr2=0x7fe3540fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 -- 172.21.15.31:0/2960199299 shutdown_connections 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.800+0000 7fe35a9d5640 1 -- 172.21.15.31:0/2960199299 wait complete. 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 Processor -- start 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 -- start start 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 0x7fe354197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe354101890 0x7fe354197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe354102250 0x7fe35419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe35410d2f0 con 0x7fe3541006a0 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe35410d170 con 0x7fe354101890 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe35a9d5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe35410d470 con 0x7fe354102250 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 0x7fe354197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 0x7fe354197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55854/0 (socket says 172.21.15.31:55854) 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 -- 172.21.15.31:0/3229669357 learned_addr learned my addr 172.21.15.31:0/3229669357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe358f72640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe354102250 0x7fe35419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 -- 172.21.15.31:0/3229669357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe354102250 msgr2=0x7fe35419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe354102250 0x7fe35419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 -- 172.21.15.31:0/3229669357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe354101890 msgr2=0x7fe354197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe354101890 0x7fe354197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 -- 172.21.15.31:0/3229669357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35419f560 con 0x7fe3541006a0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.801+0000 7fe353fff640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 0x7fe354197940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c970 tx=0x7fe34400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.802+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344007c10 con 0x7fe3541006a0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.802+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35419f850 con 0x7fe3541006a0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.802+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe344007db0 con 0x7fe3541006a0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.802+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34400f870 con 0x7fe3541006a0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.802+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35419fda0 con 0x7fe3541006a0 2024-04-25T00:30:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.803+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3541079c0 con 0x7fe3541006a0 2024-04-25T00:30:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.804+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe34400fa10 con 0x7fe3541006a0 2024-04-25T00:30:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.804+0000 7fe3517fa640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe33c0a2b90 0x7fe33c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.804+0000 7fe3537fe640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe33c0a2b90 0x7fe33c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.804+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe3440cf4b0 con 0x7fe3541006a0 2024-04-25T00:30:18.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.805+0000 7fe3537fe640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe33c0a2b90 0x7fe33c0a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe348005fd0 tx=0x7fe348005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.809+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe34409c470 con 0x7fe3541006a0 2024-04-25T00:30:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:18 smithi169 ceph-mon[33112]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-04-25T00:30:18.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:18 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2643889350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:18 smithi040 ceph-mon[32945]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-04-25T00:30:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:18 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2643889350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:18.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.993+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3540623e0 con 0x7fe3541006a0 2024-04-25T00:30:18.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.993+0000 7fe3517fa640 1 -- 172.21.15.31:0/3229669357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fe34409bd70 con 0x7fe3541006a0 2024-04-25T00:30:18.994 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe33c0a2b90 msgr2=0x7fe33c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe33c0a2b90 0x7fe33c0a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe348005fd0 tx=0x7fe348005850 comp rx=0 tx=0).stop 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 msgr2=0x7fe354197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 0x7fe354197940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c970 tx=0x7fe34400ce30 comp rx=0 tx=0).stop 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 shutdown_connections 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe354102250 0x7fe35419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe354101890 0x7fe354197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe33c0a2b90 0x7fe33c0a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 --2- 172.21.15.31:0/3229669357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3541006a0 0x7fe354197940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 >> 172.21.15.31:0/3229669357 conn(0x7fe3540fbc30 msgr2=0x7fe354104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 shutdown_connections 2024-04-25T00:30:18.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:18.995+0000 7fe35a9d5640 1 -- 172.21.15.31:0/3229669357 wait complete. 2024-04-25T00:30:19.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:30:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 -- 172.21.15.31:0/4027125053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 msgr2=0x7f7284102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 --2- 172.21.15.31:0/4027125053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284102e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7274009900 tx=0x7f727402a410 comp rx=0 tx=0).stop 2024-04-25T00:30:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 -- 172.21.15.31:0/4027125053 shutdown_connections 2024-04-25T00:30:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 --2- 172.21.15.31:0/4027125053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 0x7f7284109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 --2- 172.21.15.31:0/4027125053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284102e10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 --2- 172.21.15.31:0/4027125053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72841017a0 0x7f7284101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 -- 172.21.15.31:0/4027125053 >> 172.21.15.31:0/4027125053 conn(0x7f72840fcd10 msgr2=0x7f72840ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 -- 172.21.15.31:0/4027125053 shutdown_connections 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.111+0000 7f728b69a640 1 -- 172.21.15.31:0/4027125053 wait complete. 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 Processor -- start 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 -- start start 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72841017a0 0x7f72840754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 0x7f7284074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f728410d2d0 con 0x7f7284102990 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f728410d150 con 0x7f72841017a0 2024-04-25T00:30:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f728b69a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f728410d450 con 0x7f7284103350 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f7288c35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f7288c35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55858/0 (socket says 172.21.15.31:55858) 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f7288c35640 1 -- 172.21.15.31:0/2414826888 learned_addr learned my addr 172.21.15.31:0/2414826888 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f7289436640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72841017a0 0x7f72840754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.112+0000 7f7289c37640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 0x7f7284074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7288c35640 1 -- 172.21.15.31:0/2414826888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 msgr2=0x7f7284074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7288c35640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 0x7f7284074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7288c35640 1 -- 172.21.15.31:0/2414826888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72841017a0 msgr2=0x7f72840754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7288c35640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72841017a0 0x7f72840754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7288c35640 1 -- 172.21.15.31:0/2414826888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7284074930 con 0x7f7284102990 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7289c37640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 0x7f7284074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f7288c35640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284073b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7274005940 tx=0x7f72740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727403f030 con 0x7f7284102990 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72740318f0 con 0x7f7284102990 2024-04-25T00:30:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72841a1af0 con 0x7f7284102990 2024-04-25T00:30:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274035070 con 0x7f7284102990 2024-04-25T00:30:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.113+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72841a2000 con 0x7f7284102990 2024-04-25T00:30:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.114+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7284108be0 con 0x7f7284102990 2024-04-25T00:30:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.115+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7274042030 con 0x7f7284102990 2024-04-25T00:30:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.116+0000 7f727a7fc640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72700a2c10 0x7f72700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.116+0000 7f7289436640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72700a2c10 0x7f72700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.116+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f72740ef570 con 0x7f7284102990 2024-04-25T00:30:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.116+0000 7f7289436640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72700a2c10 0x7f72700a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72841027f0 tx=0x7f7280005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.120+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f72740bc440 con 0x7f7284102990 2024-04-25T00:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.305+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7284062340 con 0x7f7284102990 2024-04-25T00:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.305+0000 7f727a7fc640 1 -- 172.21.15.31:0/2414826888 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f72740bbd40 con 0x7f7284102990 2024-04-25T00:30:19.306 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72700a2c10 msgr2=0x7f72700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72700a2c10 0x7f72700a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72841027f0 tx=0x7f7280005f90 comp rx=0 tx=0).stop 2024-04-25T00:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 msgr2=0x7f7284073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284073b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7274005940 tx=0x7f72740042f0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 shutdown_connections 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7284103350 0x7f7284074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72700a2c10 0x7f72700a50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7284102990 0x7f7284073b30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 --2- 172.21.15.31:0/2414826888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72841017a0 0x7f72840754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 >> 172.21.15.31:0/2414826888 conn(0x7f72840fcd10 msgr2=0x7f7284105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 shutdown_connections 2024-04-25T00:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.308+0000 7f728b69a640 1 -- 172.21.15.31:0/2414826888 wait complete. 2024-04-25T00:30:19.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:30:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 -- 172.21.15.31:0/3851983625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4103360 msgr2=0x7f46a4109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 --2- 172.21.15.31:0/3851983625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4103360 0x7f46a4109bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f469c009ca0 tx=0x7f469c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 -- 172.21.15.31:0/3851983625 shutdown_connections 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 --2- 172.21.15.31:0/3851983625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4103360 0x7f46a4109bf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 --2- 172.21.15.31:0/3851983625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a41029a0 0x7f46a4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 --2- 172.21.15.31:0/3851983625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a41017b0 0x7f46a4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 -- 172.21.15.31:0/3851983625 >> 172.21.15.31:0/3851983625 conn(0x7f46a40fcd40 msgr2=0x7f46a40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 -- 172.21.15.31:0/3851983625 shutdown_connections 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 -- 172.21.15.31:0/3851983625 wait complete. 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 Processor -- start 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.422+0000 7f46a8db6640 1 -- start start 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 0x7f46a40754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a41029a0 0x7f46a4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a4103360 0x7f46a4074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f46a410d2e0 con 0x7f46a41017b0 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46a410d160 con 0x7f46a41029a0 2024-04-25T00:30:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46a410d460 con 0x7f46a4103360 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 0x7f46a40754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 0x7f46a40754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55872/0 (socket says 172.21.15.31:55872) 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 -- 172.21.15.31:0/4227175306 learned_addr learned my addr 172.21.15.31:0/4227175306 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a1d9b640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a41029a0 0x7f46a4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a2d9d640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a4103360 0x7f46a4074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 -- 172.21.15.31:0/4227175306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a4103360 msgr2=0x7f46a4074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a4103360 0x7f46a4074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 -- 172.21.15.31:0/4227175306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a41029a0 msgr2=0x7f46a4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a41029a0 0x7f46a4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 -- 172.21.15.31:0/4227175306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a4074900 con 0x7f46a41017b0 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a259c640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 0x7f46a40754f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f469400d950 tx=0x7f469400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694017070 con 0x7f46a41017b0 2024-04-25T00:30:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a41a19e0 con 0x7f46a41017b0 2024-04-25T00:30:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f469400be10 con 0x7f46a41017b0 2024-04-25T00:30:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400f620 con 0x7f46a41017b0 2024-04-25T00:30:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.423+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a1f30 con 0x7f46a41017b0 2024-04-25T00:30:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.424+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a4108ad0 con 0x7f46a41017b0 2024-04-25T00:30:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.426+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f469401a070 con 0x7f46a41017b0 2024-04-25T00:30:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.426+0000 7f46937fe640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f468c0a2b90 0x7f468c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.426+0000 7f46a1d9b640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f468c0a2b90 0x7f468c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.426+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f46940ceb20 con 0x7f46a41017b0 2024-04-25T00:30:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.427+0000 7f46a1d9b640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f468c0a2b90 0x7f468c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f46a4075260 tx=0x7f469800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:19.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.431+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f469409ba50 con 0x7f46a41017b0 2024-04-25T00:30:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.653+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a402d170 con 0x7f46a41017b0 2024-04-25T00:30:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.654+0000 7f46937fe640 1 -- 172.21.15.31:0/4227175306 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11926 (secure 0 0 0) 0x7f46a402d170 con 0x7f46a41017b0 2024-04-25T00:30:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:19.654 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:28:30.181657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24512,24517,24499,24434,24424]},"id":1}]} 2024-04-25T00:30:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f468c0a2b90 msgr2=0x7f468c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f468c0a2b90 0x7f468c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f46a4075260 tx=0x7f469800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 msgr2=0x7f46a40754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 0x7f46a40754f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f469400d950 tx=0x7f469400dd50 comp rx=0 tx=0).stop 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 shutdown_connections 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a4103360 0x7f46a4074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46a41029a0 0x7f46a4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f468c0a2b90 0x7f468c0a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 --2- 172.21.15.31:0/4227175306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a41017b0 0x7f46a40754f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 >> 172.21.15.31:0/4227175306 conn(0x7f46a40fcd40 msgr2=0x7f46a4105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 shutdown_connections 2024-04-25T00:30:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.656+0000 7f46a8db6640 1 -- 172.21.15.31:0/4227175306 wait complete. 2024-04-25T00:30:19.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:19 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4126043179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:19 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3229669357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.667 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:19 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2414826888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 -- 172.21.15.31:0/3713454226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 msgr2=0x7fed90101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 --2- 172.21.15.31:0/3713454226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90101d10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed84009900 tx=0x7fed8402a410 comp rx=0 tx=0).stop 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 -- 172.21.15.31:0/3713454226 shutdown_connections 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 --2- 172.21.15.31:0/3713454226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 0x7fed90108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 --2- 172.21.15.31:0/3713454226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90101d10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 --2- 172.21.15.31:0/3713454226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed901006a0 0x7fed90100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 -- 172.21.15.31:0/3713454226 >> 172.21.15.31:0/3713454226 conn(0x7fed900fbc30 msgr2=0x7fed900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 -- 172.21.15.31:0/3713454226 shutdown_connections 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.769+0000 7fed9677e640 1 -- 172.21.15.31:0/3713454226 wait complete. 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 Processor -- start 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 -- start start 2024-04-25T00:30:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed901006a0 0x7fed901045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 0x7fed901031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010c230 con 0x7fed90101890 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010c0b0 con 0x7fed90102250 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed9677e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010c3b0 con 0x7fed901006a0 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55888/0 (socket says 172.21.15.31:55888) 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 -- 172.21.15.31:0/1200782463 learned_addr learned my addr 172.21.15.31:0/1200782463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8ffff640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed901006a0 0x7fed901045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed94d1b640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 0x7fed901031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 -- 172.21.15.31:0/1200782463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed901006a0 msgr2=0x7fed901045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed901006a0 0x7fed901045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 -- 172.21.15.31:0/1200782463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 msgr2=0x7fed901031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 0x7fed901031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 -- 172.21.15.31:0/1200782463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90103a70 con 0x7fed90101890 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed94d1b640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 0x7fed901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.770+0000 7fed8f7fe640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90102c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fed84004770 tx=0x7fed840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.771+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8403f030 con 0x7fed90101890 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.771+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901a1b80 con 0x7fed90101890 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.771+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed84003cb0 con 0x7fed90101890 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.771+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84035070 con 0x7fed90101890 2024-04-25T00:30:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.771+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901a2090 con 0x7fed90101890 2024-04-25T00:30:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.772+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed901a22c0 con 0x7fed90101890 2024-04-25T00:30:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.773+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed84042030 con 0x7fed90101890 2024-04-25T00:30:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.774+0000 7fed8d7fa640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed780a2b90 0x7fed780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.774+0000 7fed8ffff640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed780a2b90 0x7fed780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:19.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.774+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fed840efac0 con 0x7fed90101890 2024-04-25T00:30:19.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.774+0000 7fed8ffff640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed780a2b90 0x7fed780a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fed80005fd0 tx=0x7fed80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.778+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed840bc9f0 con 0x7fed90101890 2024-04-25T00:30:19.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:19 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4126043179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:19 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3229669357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:19 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2414826888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:19 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4126043179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:19 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3229669357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:19 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2414826888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:30:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.964+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed901a2650 con 0x7fed90101890 2024-04-25T00:30:19.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.965+0000 7fed8d7fa640 1 -- 172.21.15.31:0/1200782463 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7fed840bc2f0 con 0x7fed90101890 2024-04-25T00:30:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:19.966 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:30:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed780a2b90 msgr2=0x7fed780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed780a2b90 0x7fed780a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fed80005fd0 tx=0x7fed80005850 comp rx=0 tx=0).stop 2024-04-25T00:30:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 msgr2=0x7fed90102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90102c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fed84004770 tx=0x7fed840042f0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 shutdown_connections 2024-04-25T00:30:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed90102250 0x7fed901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed780a2b90 0x7fed780a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed90101890 0x7fed90102c70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 --2- 172.21.15.31:0/1200782463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed901006a0 0x7fed901045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 >> 172.21.15.31:0/1200782463 conn(0x7fed900fbc30 msgr2=0x7fed900fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 shutdown_connections 2024-04-25T00:30:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:19.967+0000 7fed9677e640 1 -- 172.21.15.31:0/1200782463 wait complete. 2024-04-25T00:30:19.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:30:19.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:30:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 -- 172.21.15.31:0/740003161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60107460 msgr2=0x7faa601098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 --2- 172.21.15.31:0/740003161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60107460 0x7faa601098b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009ca0 tx=0x7faa5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 -- 172.21.15.31:0/740003161 shutdown_connections 2024-04-25T00:30:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 --2- 172.21.15.31:0/740003161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60107460 0x7faa601098b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 --2- 172.21.15.31:0/740003161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60104b30 0x7faa60106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 --2- 172.21.15.31:0/740003161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa60102200 0x7faa601045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.083+0000 7faa681ce640 1 -- 172.21.15.31:0/740003161 >> 172.21.15.31:0/740003161 conn(0x7faa600fbbf0 msgr2=0x7faa600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 -- 172.21.15.31:0/740003161 shutdown_connections 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 -- 172.21.15.31:0/740003161 wait complete. 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 Processor -- start 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 -- start start 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 0x7faa60197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60104b30 0x7faa60197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa60107460 0x7faa6019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa6010d2d0 con 0x7faa60102200 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa6010d150 con 0x7faa60104b30 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa681ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa6010d450 con 0x7faa60107460 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa65f6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 0x7faa60197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa65f6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 0x7faa60197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55904/0 (socket says 172.21.15.31:55904) 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa65f6a640 1 -- 172.21.15.31:0/674897281 learned_addr learned my addr 172.21.15.31:0/674897281 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.084+0000 7faa65769640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60104b30 0x7faa60197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa65f6a640 1 -- 172.21.15.31:0/674897281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa60107460 msgr2=0x7faa6019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa65f6a640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa60107460 0x7faa6019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa65f6a640 1 -- 172.21.15.31:0/674897281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60104b30 msgr2=0x7faa60197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa65f6a640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60104b30 0x7faa60197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa65f6a640 1 -- 172.21.15.31:0/674897281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6019f560 con 0x7faa60102200 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa65f6a640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 0x7faa60197940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa5800d770 tx=0x7faa5800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58017070 con 0x7faa60102200 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa58009820 con 0x7faa60102200 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6019f850 con 0x7faa60102200 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58004ba0 con 0x7faa60102200 2024-04-25T00:30:20.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.085+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6019fd50 con 0x7faa60102200 2024-04-25T00:30:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.086+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa600fd6e0 con 0x7faa60102200 2024-04-25T00:30:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.087+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa580099d0 con 0x7faa60102200 2024-04-25T00:30:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.088+0000 7faa56ffd640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faa4c0a2b90 0x7faa4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.088+0000 7faa65769640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faa4c0a2b90 0x7faa4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.088+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7faa580ceb10 con 0x7faa60102200 2024-04-25T00:30:20.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.089+0000 7faa65769640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faa4c0a2b90 0x7faa4c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa60198e50 tx=0x7faa5000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:20.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.093+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa5809ba40 con 0x7faa60102200 2024-04-25T00:30:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.277+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa6002d170 con 0x7faa60102200 2024-04-25T00:30:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.277+0000 7faa56ffd640 1 -- 172.21.15.31:0/674897281 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7faa6002d170 con 0x7faa60102200 2024-04-25T00:30:20.278 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faa4c0a2b90 msgr2=0x7faa4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faa4c0a2b90 0x7faa4c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa60198e50 tx=0x7faa5000a2d0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 msgr2=0x7faa60197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 0x7faa60197940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa5800d770 tx=0x7faa5800db70 comp rx=0 tx=0).stop 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 shutdown_connections 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa60107460 0x7faa6019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60104b30 0x7faa60197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7faa4c0a2b90 0x7faa4c0a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 --2- 172.21.15.31:0/674897281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa60102200 0x7faa60197940 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 >> 172.21.15.31:0/674897281 conn(0x7faa600fbbf0 msgr2=0x7faa60102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 shutdown_connections 2024-04-25T00:30:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.279+0000 7faa681ce640 1 -- 172.21.15.31:0/674897281 wait complete. 2024-04-25T00:30:20.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 -- 172.21.15.31:0/1736552972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f41074e0 msgr2=0x7f55f41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 --2- 172.21.15.31:0/1736552972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f41074e0 0x7f55f41098d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009ca0 tx=0x7f55ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 -- 172.21.15.31:0/1736552972 shutdown_connections 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 --2- 172.21.15.31:0/1736552972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f41074e0 0x7f55f41098d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 --2- 172.21.15.31:0/1736552972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4104bb0 0x7f55f4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 --2- 172.21.15.31:0/1736552972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f4102280 0x7f55f4104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 -- 172.21.15.31:0/1736552972 >> 172.21.15.31:0/1736552972 conn(0x7f55f40fbc30 msgr2=0x7f55f40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 -- 172.21.15.31:0/1736552972 shutdown_connections 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.395+0000 7f55fae15640 1 -- 172.21.15.31:0/1736552972 wait complete. 2024-04-25T00:30:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 Processor -- start 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 -- start start 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4102280 0x7f55f41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 0x7f55f4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f41074e0 0x7f55f419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410d2f0 con 0x7f55f4104bb0 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410d170 con 0x7f55f4102280 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55fae15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410d470 con 0x7f55f41074e0 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 0x7f55f4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 0x7f55f4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55910/0 (socket says 172.21.15.31:55910) 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 -- 172.21.15.31:0/3543319132 learned_addr learned my addr 172.21.15.31:0/3543319132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f93b2640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f41074e0 0x7f55f419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f8bb1640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4102280 0x7f55f41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 -- 172.21.15.31:0/3543319132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f41074e0 msgr2=0x7f55f419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f41074e0 0x7f55f419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 -- 172.21.15.31:0/3543319132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4102280 msgr2=0x7f55f41979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4102280 0x7f55f41979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 -- 172.21.15.31:0/3543319132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f419f5d0 con 0x7f55f4104bb0 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f8bb1640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4102280 0x7f55f41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f3fff640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 0x7f55f4197ef0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005fd0 tx=0x7f55e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.396+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0017070 con 0x7f55f4104bb0 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.397+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f419f8c0 con 0x7f55f4104bb0 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.397+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55e0009c30 con 0x7f55f4104bb0 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.397+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0003d40 con 0x7f55f4104bb0 2024-04-25T00:30:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.397+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f419fe10 con 0x7f55f4104bb0 2024-04-25T00:30:20.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.398+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f40fd720 con 0x7f55f4104bb0 2024-04-25T00:30:20.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.399+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f55e0003ee0 con 0x7f55f4104bb0 2024-04-25T00:30:20.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.399+0000 7f55f1ffb640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55e40a2b90 0x7f55e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.399+0000 7f55f8bb1640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55e40a2b90 0x7f55e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.399+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f55e00cede0 con 0x7f55f4104bb0 2024-04-25T00:30:20.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.400+0000 7f55f8bb1640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55e40a2b90 0x7f55e40a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009b40 tx=0x7f55e8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.404+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f55e0020030 con 0x7f55f4104bb0 2024-04-25T00:30:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.589+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55f402d170 con 0x7f55f4104bb0 2024-04-25T00:30:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.590+0000 7f55f1ffb640 1 -- 172.21.15.31:0/3543319132 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f55e009bd10 con 0x7f55f4104bb0 2024-04-25T00:30:20.591 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:30:20.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55e40a2b90 msgr2=0x7f55e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55e40a2b90 0x7f55e40a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009b40 tx=0x7f55e8005e50 comp rx=0 tx=0).stop 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 msgr2=0x7f55f4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 0x7f55f4197ef0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005fd0 tx=0x7f55e000dde0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 shutdown_connections 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f41074e0 0x7f55f419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f55e40a2b90 0x7f55e40a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f4104bb0 0x7f55f4197ef0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 --2- 172.21.15.31:0/3543319132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f4102280 0x7f55f41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 >> 172.21.15.31:0/3543319132 conn(0x7f55f40fbc30 msgr2=0x7f55f4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 shutdown_connections 2024-04-25T00:30:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.592+0000 7f55fae15640 1 -- 172.21.15.31:0/3543319132 wait complete. 2024-04-25T00:30:20.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:20 smithi031 ceph-mon[29167]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:30:20.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:20 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4227175306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:20.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:20 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1200782463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:20.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:20 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/674897281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:30:20.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 -- 172.21.15.31:0/2133927386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 msgr2=0x7fcabc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 --2- 172.21.15.31:0/2133927386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc108ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ca0 tx=0x7fcab402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 -- 172.21.15.31:0/2133927386 shutdown_connections 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 --2- 172.21.15.31:0/2133927386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc108ae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 --2- 172.21.15.31:0/2133927386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc101890 0x7fcabc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 --2- 172.21.15.31:0/2133927386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc1006a0 0x7fcabc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 -- 172.21.15.31:0/2133927386 >> 172.21.15.31:0/2133927386 conn(0x7fcabc0fbc30 msgr2=0x7fcabc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 -- 172.21.15.31:0/2133927386 shutdown_connections 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 -- 172.21.15.31:0/2133927386 wait complete. 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 Processor -- start 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.709+0000 7fcac1412640 1 -- start start 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc1006a0 0x7fcabc104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc101890 0x7fcabc102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10c1b0 con 0x7fcabc102250 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10c030 con 0x7fcabc101890 2024-04-25T00:30:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10c330 con 0x7fcabc1006a0 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc1031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55926/0 (socket says 172.21.15.31:55926) 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 -- 172.21.15.31:0/2602443516 learned_addr learned my addr 172.21.15.31:0/2602443516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcab3fff640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc101890 0x7fcabc102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabaffd640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc1006a0 0x7fcabc104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 -- 172.21.15.31:0/2602443516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc1006a0 msgr2=0x7fcabc104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc1006a0 0x7fcabc104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 -- 172.21.15.31:0/2602443516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc101890 msgr2=0x7fcabc102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc101890 0x7fcabc102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 -- 172.21.15.31:0/2602443516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc103a40 con 0x7fcabc102250 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcabb7fe640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc1031b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009dc0 tx=0x7fcab4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4032ec0 con 0x7fcabc102250 2024-04-25T00:30:20.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab40044f0 con 0x7fcabc102250 2024-04-25T00:30:20.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1a1a10 con 0x7fcabc102250 2024-04-25T00:30:20.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4030d40 con 0x7fcabc102250 2024-04-25T00:30:20.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.710+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a1f60 con 0x7fcabc102250 2024-04-25T00:30:20.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.711+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc1a2190 con 0x7fcabc102250 2024-04-25T00:30:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.713+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcab4004040 con 0x7fcabc102250 2024-04-25T00:30:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.713+0000 7fcab8ff9640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcaa80a2d30 0x7fcaa80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.713+0000 7fcabaffd640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcaa80a2d30 0x7fcaa80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.713+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fcab40f0670 con 0x7fcabc102250 2024-04-25T00:30:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.714+0000 7fcabaffd640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcaa80a2d30 0x7fcaa80a51f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005fd0 tx=0x7fcaac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:20.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.718+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcab40bd5a0 con 0x7fcabc102250 2024-04-25T00:30:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:20 smithi169 ceph-mon[33112]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:30:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:20 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4227175306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:20 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1200782463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:20 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/674897281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:30:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:20 smithi040 ceph-mon[32945]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:30:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:20 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4227175306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:20 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1200782463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:20 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/674897281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:30:20.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.904+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcabc1a2480 con 0x7fcabc102250 2024-04-25T00:30:20.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.904+0000 7fcab8ff9640 1 -- 172.21.15.31:0/2602443516 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fcab40bcea0 con 0x7fcabc102250 2024-04-25T00:30:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:30:20.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcaa80a2d30 msgr2=0x7fcaa80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcaa80a2d30 0x7fcaa80a51f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005fd0 tx=0x7fcaac005850 comp rx=0 tx=0).stop 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 msgr2=0x7fcabc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc1031b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009dc0 tx=0x7fcab4003ba0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 shutdown_connections 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fcaa80a2d30 0x7fcaa80a51f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcabc102250 0x7fcabc1031b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc101890 0x7fcabc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 --2- 172.21.15.31:0/2602443516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc1006a0 0x7fcabc104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:20.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 >> 172.21.15.31:0/2602443516 conn(0x7fcabc0fbc30 msgr2=0x7fcabc0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 shutdown_connections 2024-04-25T00:30:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:20.906+0000 7fcac1412640 1 -- 172.21.15.31:0/2602443516 wait complete. 2024-04-25T00:30:20.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:30:21.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 -- 172.21.15.31:0/1514911837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8101890 msgr2=0x7f32d8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 --2- 172.21.15.31:0/1514911837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8101890 0x7f32d8101d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009900 tx=0x7f32cc02a410 comp rx=0 tx=0).stop 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 -- 172.21.15.31:0/1514911837 shutdown_connections 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 --2- 172.21.15.31:0/1514911837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8102250 0x7f32d8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 --2- 172.21.15.31:0/1514911837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8101890 0x7f32d8101d10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 --2- 172.21.15.31:0/1514911837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 0x7f32d8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 -- 172.21.15.31:0/1514911837 >> 172.21.15.31:0/1514911837 conn(0x7f32d80fbc30 msgr2=0x7f32d80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 -- 172.21.15.31:0/1514911837 shutdown_connections 2024-04-25T00:30:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.019+0000 7f32dcad4640 1 -- 172.21.15.31:0/1514911837 wait complete. 2024-04-25T00:30:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 Processor -- start 2024-04-25T00:30:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 -- start start 2024-04-25T00:30:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 0x7f32d8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8101890 0x7f32d8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 0x7f32d819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810c1b0 con 0x7f32d8102250 2024-04-25T00:30:21.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810c030 con 0x7f32d81006a0 2024-04-25T00:30:21.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32dcad4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810c330 con 0x7f32d8101890 2024-04-25T00:30:21.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 0x7f32d819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 0x7f32d819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55946/0 (socket says 172.21.15.31:55946) 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 -- 172.21.15.31:0/516417450 learned_addr learned my addr 172.21.15.31:0/516417450 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d659c640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 0x7f32d8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d5d9b640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8101890 0x7f32d8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 -- 172.21.15.31:0/516417450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8101890 msgr2=0x7f32d8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8101890 0x7f32d8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 -- 172.21.15.31:0/516417450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 msgr2=0x7f32d8197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 0x7f32d8197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.020+0000 7f32d6d9d640 1 -- 172.21.15.31:0/516417450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d819f620 con 0x7f32d8102250 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32d5d9b640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8101890 0x7f32d8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32d659c640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 0x7f32d8197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32d6d9d640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 0x7f32d819efb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f32d000cad0 tx=0x7f32d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000ae20 con 0x7f32d8102250 2024-04-25T00:30:21.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d0005020 con 0x7f32d8102250 2024-04-25T00:30:21.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d819f910 con 0x7f32d8102250 2024-04-25T00:30:21.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000f6b0 con 0x7f32d8102250 2024-04-25T00:30:21.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.021+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d819fe60 con 0x7f32d8102250 2024-04-25T00:30:21.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.022+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d81a00f0 con 0x7f32d8102250 2024-04-25T00:30:21.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.023+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32d00051d0 con 0x7f32d8102250 2024-04-25T00:30:21.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.024+0000 7f32c77fe640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f32c00a2c10 0x7f32c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.024+0000 7f32d659c640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f32c00a2c10 0x7f32c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.024+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f32d00cf510 con 0x7f32d8102250 2024-04-25T00:30:21.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.024+0000 7f32d659c640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f32c00a2c10 0x7f32c00a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f32c80058c0 tx=0x7f32c8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.027+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32d0014030 con 0x7f32d8102250 2024-04-25T00:30:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.253+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32d81a02c0 con 0x7f32d8102250 2024-04-25T00:30:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.254+0000 7f32c77fe640 1 -- 172.21.15.31:0/516417450 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11926 (secure 0 0 0) 0x7f32d009c4c0 con 0x7f32d8102250 2024-04-25T00:30:21.255 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:21.255 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:28:30.181657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24512,24517,24499,24434,24424]},"id":1}]} 2024-04-25T00:30:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f32c00a2c10 msgr2=0x7f32c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f32c00a2c10 0x7f32c00a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f32c80058c0 tx=0x7f32c8005850 comp rx=0 tx=0).stop 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 msgr2=0x7f32d819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 0x7f32d819efb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f32d000cad0 tx=0x7f32d000cf90 comp rx=0 tx=0).stop 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 shutdown_connections 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f32c00a2c10 0x7f32c00a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32d8102250 0x7f32d819efb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d8101890 0x7f32d8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 --2- 172.21.15.31:0/516417450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d81006a0 0x7f32d8197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 >> 172.21.15.31:0/516417450 conn(0x7f32d80fbc30 msgr2=0x7f32d8069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 shutdown_connections 2024-04-25T00:30:21.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.256+0000 7f32dcad4640 1 -- 172.21.15.31:0/516417450 wait complete. 2024-04-25T00:30:21.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 -- 172.21.15.31:0/3482440392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c102980 msgr2=0x7f9f0c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3482440392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c102980 0x7f9f0c102e00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009900 tx=0x7f9f0002a410 comp rx=0 tx=0).stop 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 -- 172.21.15.31:0/3482440392 shutdown_connections 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3482440392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 0x7f9f0c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3482440392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c102980 0x7f9f0c102e00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3482440392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c101790 0x7f9f0c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 -- 172.21.15.31:0/3482440392 >> 172.21.15.31:0/3482440392 conn(0x7f9f0c0fcd20 msgr2=0x7f9f0c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 -- 172.21.15.31:0/3482440392 shutdown_connections 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 -- 172.21.15.31:0/3482440392 wait complete. 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 Processor -- start 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.370+0000 7f9f123ee640 1 -- start start 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 0x7f9f0c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c102980 0x7f9f0c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 0x7f9f0c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10d2a0 con 0x7f9f0c101790 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10d120 con 0x7f9f0c102980 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10d420 con 0x7f9f0c103340 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 0x7f9f0c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 0x7f9f0c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55974/0 (socket says 172.21.15.31:55974) 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 -- 172.21.15.31:0/3959279747 learned_addr learned my addr 172.21.15.31:0/3959279747 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f1098b640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 0x7f9f0c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0b7fe640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c102980 0x7f9f0c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 -- 172.21.15.31:0/3959279747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 msgr2=0x7f9f0c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 0x7f9f0c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 -- 172.21.15.31:0/3959279747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c102980 msgr2=0x7f9f0c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c102980 0x7f9f0c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 -- 172.21.15.31:0/3959279747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c19f5f0 con 0x7f9f0c101790 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0b7fe640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c102980 0x7f9f0c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f1098b640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 0x7f9f0c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f0bfff640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 0x7f9f0c1979d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00d770 tx=0x7f9efc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc017070 con 0x7f9f0c101790 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c19f8e0 con 0x7f9f0c101790 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9efc009c30 con 0x7f9f0c101790 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc003d00 con 0x7f9f0c101790 2024-04-25T00:30:21.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.371+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c19fe30 con 0x7f9f0c101790 2024-04-25T00:30:21.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.372+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f0c108b20 con 0x7f9f0c101790 2024-04-25T00:30:21.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.373+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9efc003ea0 con 0x7f9f0c101790 2024-04-25T00:30:21.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.374+0000 7f9f097fa640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ef40a2b90 0x7f9ef40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.374+0000 7f9f0b7fe640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ef40a2b90 0x7f9ef40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.374+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9efc0cee40 con 0x7f9f0c101790 2024-04-25T00:30:21.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.375+0000 7f9f0b7fe640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ef40a2b90 0x7f9ef40a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f00006010 tx=0x7f9f00000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.379+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9efc09bd70 con 0x7f9f0c101790 2024-04-25T00:30:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3543319132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:30:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2602443516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:30:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:21 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/516417450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.568+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f0c0623e0 con 0x7f9f0c101790 2024-04-25T00:30:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.570+0000 7f9f097fa640 1 -- 172.21.15.31:0/3959279747 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7f9efc009de0 con 0x7f9f0c101790 2024-04-25T00:30:21.571 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:30:21.571 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:30:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.572+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ef40a2b90 msgr2=0x7f9ef40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.572+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ef40a2b90 0x7f9ef40a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f00006010 tx=0x7f9f00000bc0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.572+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 msgr2=0x7f9f0c1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.572+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 0x7f9f0c1979d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00d770 tx=0x7f9efc00dc30 comp rx=0 tx=0).stop 2024-04-25T00:30:21.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 shutdown_connections 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f0c103340 0x7f9f0c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0c102980 0x7f9f0c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9ef40a2b90 0x7f9ef40a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 --2- 172.21.15.31:0/3959279747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101790 0x7f9f0c1979d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 >> 172.21.15.31:0/3959279747 conn(0x7f9f0c0fcd20 msgr2=0x7f9f0c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 shutdown_connections 2024-04-25T00:30:21.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.573+0000 7f9f123ee640 1 -- 172.21.15.31:0/3959279747 wait complete. 2024-04-25T00:30:21.585 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:28:23.753095+0000', 'modified': '2024-04-25T00:28:30.181657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24517, 'mds_1': 24512, 'mds_2': 24499, 'mds_3': 24434, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/2045809592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 2045809592}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 2045809592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/3504624607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3504624607}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3504624607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.169:6833/2672070962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 2672070962}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 2672070962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'j', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.31:6835/310214957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 310214957}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 310214957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/653343360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 653343360}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 653343360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6837/3765244618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6836', 'nonce': 3765244618}, {'type': 'v1', 'addr': '172.21.15.40:6837', 'nonce': 3765244618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/3258138877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3258138877}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3258138877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6837/1684471092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1684471092}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1684471092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6839/2724938586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 2724938586}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 2724938586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.169:6839/3109017383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 3109017383}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 3109017383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24517, 'qdb_cluster': [24512, 24517, 24499, 24434, 24424]} 2024-04-25T00:30:21.585 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T00:30:21.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-25T00:30:21.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 -- 172.21.15.31:0/4136409799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c1074e0 msgr2=0x7f7f7c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 --2- 172.21.15.31:0/4136409799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c1074e0 0x7f7f7c1098d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009ca0 tx=0x7f7f7402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 -- 172.21.15.31:0/4136409799 shutdown_connections 2024-04-25T00:30:21.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 --2- 172.21.15.31:0/4136409799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c1074e0 0x7f7f7c1098d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 --2- 172.21.15.31:0/4136409799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 --2- 172.21.15.31:0/4136409799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c102280 0x7f7f7c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 -- 172.21.15.31:0/4136409799 >> 172.21.15.31:0/4136409799 conn(0x7f7f7c0fbc30 msgr2=0x7f7f7c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 -- 172.21.15.31:0/4136409799 shutdown_connections 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.687+0000 7f7f84215640 1 -- 172.21.15.31:0/4136409799 wait complete. 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 Processor -- start 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 -- start start 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c102280 0x7f7f7c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c1074e0 0x7f7f7c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c10d340 con 0x7f7f7c104bb0 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c10d1c0 con 0x7f7f7c102280 2024-04-25T00:30:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f84215640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c10d4c0 con 0x7f7f7c1074e0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55994/0 (socket says 172.21.15.31:55994) 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 -- 172.21.15.31:0/3419715432 learned_addr learned my addr 172.21.15.31:0/3419715432 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c102280 0x7f7f7c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f827b2640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c1074e0 0x7f7f7c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 -- 172.21.15.31:0/3419715432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c1074e0 msgr2=0x7f7f7c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c1074e0 0x7f7f7c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 -- 172.21.15.31:0/3419715432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c102280 msgr2=0x7f7f7c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c102280 0x7f7f7c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f817b0640 1 -- 172.21.15.31:0/3419715432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c19f620 con 0x7f7f7c104bb0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.688+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c102280 0x7f7f7c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.689+0000 7f7f817b0640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c197f40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00d770 tx=0x7f7f6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.689+0000 7f7f72ffd640 1 -- 172.21.15.31:0/3419715432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c017070 con 0x7f7f7c104bb0 2024-04-25T00:30:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.689+0000 7f7f72ffd640 1 -- 172.21.15.31:0/3419715432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f6c009c30 con 0x7f7f7c104bb0 2024-04-25T00:30:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.689+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c19f910 con 0x7f7f7c104bb0 2024-04-25T00:30:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.689+0000 7f7f72ffd640 1 -- 172.21.15.31:0/3419715432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00f620 con 0x7f7f7c104bb0 2024-04-25T00:30:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.689+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c19fe60 con 0x7f7f7c104bb0 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.691+0000 7f7f72ffd640 1 -- 172.21.15.31:0/3419715432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f6c01a070 con 0x7f7f7c104bb0 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.691+0000 7f7f86ffe740 1 Processor -- start 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 -- start start 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f842d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f8432930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f8435430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5603f8390f00 con 0x5603f842d440 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5603f8339750 con 0x5603f8432fa0 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5603f838cec0 con 0x5603f842de30 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f72ffd640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7f680a2b90 0x7f7f680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f842d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f842d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56004/0 (socket says 172.21.15.31:56004) 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f81fb1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f8432930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 -- 172.21.15.31:0/857533323 learned_addr learned my addr 172.21.15.31:0/857533323 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7f680a2b90 0x7f7f680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f72ffd640 1 -- 172.21.15.31:0/3419715432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7f6c0cecb0 con 0x7f7f7c104bb0 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f817b0640 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f8435430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 -- 172.21.15.31:0/857533323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 msgr2=0x5603f8432930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f8432930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 -- 172.21.15.31:0/857533323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 msgr2=0x5603f8435430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f8435430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f827b2640 1 -- 172.21.15.31:0/857533323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f80a52d0 con 0x5603f842d440 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.692+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f8432930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f827b2640 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f842d860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7f74032d00 tx=0x7f7f74032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c505ac13dab512b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f827b2640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7f680a2b90 0x7f7f680a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006010 tx=0x7f7f74042000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f67fff640 1 -- 172.21.15.31:0/857533323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7404e070 con 0x5603f842d440 2024-04-25T00:30:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f67fff640 1 -- 172.21.15.31:0/857533323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f74036050 con 0x5603f842d440 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f67fff640 1 -- 172.21.15.31:0/857533323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74030820 con 0x5603f842d440 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f86ffe740 1 -- 172.21.15.31:0/857533323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 msgr2=0x5603f842d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.693+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f842d860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7f74032d00 tx=0x7f7f74032d30 comp rx=0 tx=0).stop 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- 172.21.15.31:0/857533323 shutdown_connections 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f8435430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f8432930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/857533323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f842d860 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- 172.21.15.31:0/857533323 >> 172.21.15.31:0/857533323 conn(0x5603f8428ab0 msgr2=0x5603f8428eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- 172.21.15.31:0/857533323 shutdown_connections 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- 172.21.15.31:0/857533323 wait complete. 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- start start 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 Processor -- start 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842d440 0x5603f8437920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f8437e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f843ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5603f8435ef0 con 0x5603f842de30 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5603f834ccf0 con 0x5603f8432fa0 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5603f8436070 con 0x5603f842d440 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f86ffe740 10 client.0 _ll_register_callbacks cb 0x5603f834c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f817b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f8437e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f817b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f8437e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56014/0 (socket says 172.21.15.31:56014) 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.694+0000 7f7f817b0640 1 -- 172.21.15.31:0/2759002316 learned_addr learned my addr 172.21.15.31:0/2759002316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842d440 0x5603f8437920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f817b0640 1 -- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842d440 msgr2=0x5603f8437920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f817b0640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842d440 0x5603f8437920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f817b0640 1 -- 172.21.15.31:0/2759002316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 msgr2=0x5603f843ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f827b2640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f843ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f817b0640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f843ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f817b0640 1 -- 172.21.15.31:0/2759002316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f816f6b0 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f817b0640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f8437e60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c09b180 tx=0x7f7f6c09b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cec2a9096208ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f657fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f657fa640 1 -- 172.21.15.31:0/2759002316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c021040 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f657fa640 1 -- 172.21.15.31:0/2759002316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7f6c0b4220 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f74036050 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f657fa640 1 -- 172.21.15.31:0/2759002316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f6c09b480 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f86ffe740 10 client.15267 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603f8441ed0 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.695+0000 7f7f657fa640 1 -- 172.21.15.31:0/2759002316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f74036050 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f657fa640 1 -- 172.21.15.31:0/2759002316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f6c0ccbe0 con 0x5603f842de30 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f86ffe740 10 client.15267 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f86ffe740 10 client.15267 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f86ffe740 20 client.15267 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f8445980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f86ffe740 4 client.15267 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 --> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f839e440 con 0x5603f8444510 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f8445980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f657fa640 10 client.15267 ms_handle_connect on v2:172.21.15.169:6832/2672070962 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.696+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f8445980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66f0f5b00f61c6b6 server_cookie=1a549d03e8cecbd1 in_seq=0 out_seq=0 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f657fa640 1 -- 172.21.15.31:0/2759002316 <== mds.4 v2:172.21.15.169:6832/2672070962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603f839e440 con 0x5603f8444510 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f657fa640 10 client.15267 handle_command_reply: tid=0 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 client.15267 shutdown 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 msgr2=0x5603f8437e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f8437e60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c09b180 tx=0x7f7f6c09b370 comp rx=0 tx=0).stop 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 shutdown_connections 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f8445980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f843ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f8437e60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f657fa640 0 client.15267 ms_handle_reset on v2:172.21.15.169:6832/2672070962 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2759002316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842d440 0x5603f8437920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.697+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 >> 172.21.15.31:0/2759002316 conn(0x5603f8428ab0 msgr2=0x5603f8427ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.698+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 shutdown_connections 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.698+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2759002316 wait complete. 2024-04-25T00:30:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.698+0000 7f7f86ffe740 20 client.15267 trim_cache size 0 max 16384 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 Processor -- start 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 -- start start 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f8428260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f843edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f842c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5603f8443cc0 con 0x5603f842d440 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5603f8444390 con 0x5603f8432fa0 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5603f8444a60 con 0x5603f842de30 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f8428260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f8428260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56026/0 (socket says 172.21.15.31:56026) 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 -- 172.21.15.31:0/2876658227 learned_addr learned my addr 172.21.15.31:0/2876658227 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 -- 172.21.15.31:0/2876658227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 msgr2=0x5603f843edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f843edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 -- 172.21.15.31:0/2876658227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 msgr2=0x5603f842c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f827b2640 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f842c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f842c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 -- 172.21.15.31:0/2876658227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f842e6c0 con 0x5603f842d440 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f827b2640 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f842c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.716+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f8428260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f780075d0 tx=0x7f7f78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc332d6a170c751e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f677fe640 1 -- 172.21.15.31:0/2876658227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78007d40 con 0x5603f842d440 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f677fe640 1 -- 172.21.15.31:0/2876658227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f7801b590 con 0x5603f842d440 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f677fe640 1 -- 172.21.15.31:0/2876658227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78024820 con 0x5603f842d440 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2876658227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 msgr2=0x5603f8428260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f8428260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f780075d0 tx=0x7f7f78008040 comp rx=0 tx=0).stop 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2876658227 shutdown_connections 2024-04-25T00:30:21.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f8432fa0 0x5603f842c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f842de30 0x5603f843edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2876658227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842d440 0x5603f8428260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2876658227 >> 172.21.15.31:0/2876658227 conn(0x5603f8428ab0 msgr2=0x5603f842b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2876658227 shutdown_connections 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2876658227 wait complete. 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 -- start start 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.717+0000 7f7f86ffe740 1 Processor -- start 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f842d440 0x5603f843ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f843b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f8432fa0 0x5603f843e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5603f8445800 con 0x5603f842de30 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5603f8445130 con 0x5603f842d440 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5603f8445ed0 con 0x5603f8432fa0 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f86ffe740 10 client.0 _ll_register_callbacks cb 0x7f7f6c09afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f843b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f843b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56040/0 (socket says 172.21.15.31:56040) 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 -- 172.21.15.31:0/2412157153 learned_addr learned my addr 172.21.15.31:0/2412157153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f842d440 0x5603f843ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f827b2640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f8432fa0 0x5603f843e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 -- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f8432fa0 msgr2=0x5603f843e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f8432fa0 0x5603f843e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 -- 172.21.15.31:0/2412157153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f842d440 msgr2=0x5603f843ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f842d440 0x5603f843ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 -- 172.21.15.31:0/2412157153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f843e890 con 0x5603f842de30 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f842d440 0x5603f843ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f817b0640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f843b280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00d740 tx=0x7f7f6c097d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58066a3cba39d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.718+0000 7f7f657fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f657fa640 1 -- 172.21.15.31:0/2412157153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c021070 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f657fa640 1 -- 172.21.15.31:0/2412157153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7f6c0664b0 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7801b590 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f657fa640 1 -- 172.21.15.31:0/2412157153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f6c09f830 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 10 client.15273 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603f84aac90 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f657fa640 1 -- 172.21.15.31:0/2412157153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f7801b590 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f657fa640 1 -- 172.21.15.31:0/2412157153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f6c07b020 con 0x5603f842de30 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 10 client.15273 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 10 client.15273 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 20 client.15273 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f84ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 4 client.15273 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 --> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] -- command(tid 0: {"prefix": "status"}) -- 0x5603f8398790 con 0x5603f8444510 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.719+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f84ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f81fb1640 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f84ae460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3890bbbe6a10ba2 server_cookie=2d54dd1987aca7ab in_seq=0 out_seq=0 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f657fa640 10 client.15273 ms_handle_connect on v2:172.21.15.169:6832/2672070962 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f657fa640 1 -- 172.21.15.31:0/2412157153 <== mds.4 v2:172.21.15.169:6832/2672070962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5603f8398790 con 0x5603f8444510 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f657fa640 10 client.15273 handle_command_reply: tid=0 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 client.15273 shutdown 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 msgr2=0x5603f843b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f843b280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00d740 tx=0x7f7f6c097d20 comp rx=0 tx=0).stop 2024-04-25T00:30:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 shutdown_connections 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603f8432fa0 0x5603f843e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603f842de30 0x5603f843b280 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.169:6832/2672070962,v1:172.21.15.169:6833/2672070962] conn(0x5603f8444510 0x5603f84ae460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 --2- 172.21.15.31:0/2412157153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603f842d440 0x5603f843ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f657fa640 0 client.15273 ms_handle_reset on v2:172.21.15.169:6832/2672070962 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.720+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 >> 172.21.15.31:0/2412157153 conn(0x5603f8428ab0 msgr2=0x5603f8427250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 shutdown_connections 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f86ffe740 1 -- 172.21.15.31:0/2412157153 wait complete. 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f86ffe740 20 client.15273 trim_cache size 0 max 16384 2024-04-25T00:30:21.728 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24424, 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 112.5550878, 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 155.74229337200001 2024-04-25T00:30:21.729 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:30:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7f680a2b90 msgr2=0x7f7f680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7f680a2b90 0x7f7f680a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006010 tx=0x7f7f74042000 comp rx=0 tx=0).stop 2024-04-25T00:30:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 msgr2=0x7f7f7c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c197f40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00d770 tx=0x7f7f6c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:30:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 shutdown_connections 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c1074e0 0x7f7f7c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7f680a2b90 0x7f7f680a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f7c104bb0 0x7f7f7c197f40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 --2- 172.21.15.31:0/3419715432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c102280 0x7f7f7c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 >> 172.21.15.31:0/3419715432 conn(0x7f7f7c0fbc30 msgr2=0x7f7f7c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 shutdown_connections 2024-04-25T00:30:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.721+0000 7f7f84215640 1 -- 172.21.15.31:0/3419715432 wait complete. 2024-04-25T00:30:21.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.j status 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 -- 172.21.15.31:0/2641941876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 msgr2=0x7f0648100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 --2- 172.21.15.31:0/2641941876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648100aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0638009900 tx=0x7f063802a410 comp rx=0 tx=0).stop 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 -- 172.21.15.31:0/2641941876 shutdown_connections 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 --2- 172.21.15.31:0/2641941876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0648102230 0x7f0648108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 --2- 172.21.15.31:0/2641941876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648101870 0x7f0648101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 --2- 172.21.15.31:0/2641941876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648100aa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 -- 172.21.15.31:0/2641941876 >> 172.21.15.31:0/2641941876 conn(0x7f06480fbbf0 msgr2=0x7f06480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.834+0000 7f064cb4f640 1 -- 172.21.15.31:0/2641941876 shutdown_connections 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 -- 172.21.15.31:0/2641941876 wait complete. 2024-04-25T00:30:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 Processor -- start 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 -- start start 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0648101870 0x7f0648197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648102230 0x7f064819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f064810c1b0 con 0x7f0648100680 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f064810c030 con 0x7f0648102230 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.835+0000 7f064cb4f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f064810c330 con 0x7f0648101870 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56046/0 (socket says 172.21.15.31:56046) 2024-04-25T00:30:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 -- 172.21.15.31:0/4084296126 learned_addr learned my addr 172.21.15.31:0/4084296126 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f0645d9b640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0648101870 0x7f0648197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f0646d9d640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648102230 0x7f064819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 -- 172.21.15.31:0/4084296126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0648101870 msgr2=0x7f0648197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0648101870 0x7f0648197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 -- 172.21.15.31:0/4084296126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648102230 msgr2=0x7f064819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648102230 0x7f064819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 -- 172.21.15.31:0/4084296126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064819f620 con 0x7f0648100680 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f0646d9d640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648102230 0x7f064819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064659c640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648197a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f06380314e0 tx=0x7f0638004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f06377fe640 1 -- 172.21.15.31:0/4084296126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063803f030 con 0x7f0648100680 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f06377fe640 1 -- 172.21.15.31:0/4084296126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0638003aa0 con 0x7f0648100680 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064819f8b0 con 0x7f0648100680 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f06377fe640 1 -- 172.21.15.31:0/4084296126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638035070 con 0x7f0648100680 2024-04-25T00:30:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.836+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064819fda0 con 0x7f0648100680 2024-04-25T00:30:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.838+0000 7f06377fe640 1 -- 172.21.15.31:0/4084296126 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0638042030 con 0x7f0648100680 2024-04-25T00:30:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 Processor -- start 2024-04-25T00:30:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 -- start start 2024-04-25T00:30:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f06377fe640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f06300a2b90 0x7f06300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098eec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098eef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558098e4af00 con 0x558098ee7e30 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558098df3750 con 0x558098ee7440 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064f85a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558098e46ec0 con 0x558098eecfa0 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f0646d9d640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f06300a2b90 0x7f06300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f06377fe640 1 -- 172.21.15.31:0/4084296126 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f06380ef8a0 con 0x7f0648100680 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f0645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098eec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f0645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098eec930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56056/0 (socket says 172.21.15.31:56056) 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f0645d9b640 1 -- 172.21.15.31:0/1309449696 learned_addr learned my addr 172.21.15.31:0/1309449696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.839+0000 7f064659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0645d9b640 1 -- 172.21.15.31:0/1309449696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 msgr2=0x558098eef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0645d9b640 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098eef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0645d9b640 1 -- 172.21.15.31:0/1309449696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 msgr2=0x558098ee7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0645d9b640 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0645d9b640 1 -- 172.21.15.31:0/1309449696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558098b5f2d0 con 0x558098ee7e30 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f064659c640 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0646d9d640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f06300a2b90 0x7f06300a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0640005fd0 tx=0x7f0640009b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0645d9b640 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098eec930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f063c009ba0 tx=0x7f063c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225ba26ed29c25fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0634ff9640 1 -- 172.21.15.31:0/1309449696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c012450 con 0x558098ee7e30 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0634ff9640 1 -- 172.21.15.31:0/1309449696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f063c012aa0 con 0x558098ee7e30 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.840+0000 7f0634ff9640 1 -- 172.21.15.31:0/1309449696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c00fa80 con 0x558098ee7e30 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 -- 172.21.15.31:0/1309449696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 msgr2=0x558098eec930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098eec930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f063c009ba0 tx=0x7f063c0117b0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 -- 172.21.15.31:0/1309449696 shutdown_connections 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098eef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098eec930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 --2- 172.21.15.31:0/1309449696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 -- 172.21.15.31:0/1309449696 >> 172.21.15.31:0/1309449696 conn(0x558098ee2ab0 msgr2=0x558098ee2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 -- 172.21.15.31:0/1309449696 shutdown_connections 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 -- 172.21.15.31:0/1309449696 wait complete. 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 -- start start 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.841+0000 7f064f85a740 1 Processor -- start 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 0x558098ef1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7e30 0x558098ef1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098ef8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558098eeff50 con 0x558098ee7440 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558098e06cf0 con 0x558098ee7e30 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558098ef00d0 con 0x558098eecfa0 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 10 client.0 _ll_register_callbacks cb 0x558098e066e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 0x558098ef1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 0x558098ef1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56068/0 (socket says 172.21.15.31:56068) 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 -- 172.21.15.31:0/332926670 learned_addr learned my addr 172.21.15.31:0/332926670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f0646d9d640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098ef8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 -- 172.21.15.31:0/332926670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 msgr2=0x558098ef8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098ef8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 -- 172.21.15.31:0/332926670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7e30 msgr2=0x558098ef1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7e30 0x558098ef1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 -- 172.21.15.31:0/332926670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558098c296b0 con 0x558098ee7440 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f0646d9d640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098ef8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064659c640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 0x558098ef1840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0638032fd0 tx=0x7f06380319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd726e6b91ce7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f062dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f062dffb640 1 -- 172.21.15.31:0/332926670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638049040 con 0x558098ee7440 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f062dffb640 1 -- 172.21.15.31:0/332926670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f06380dc0b0 con 0x558098ee7440 2024-04-25T00:30:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f063c012aa0 con 0x558098ee7440 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f062dffb640 1 -- 172.21.15.31:0/332926670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06380bca60 con 0x558098ee7440 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 10 client.15285 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.842+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558098efbda0 con 0x558098ee7440 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f062dffb640 1 -- 172.21.15.31:0/332926670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f063c012aa0 con 0x558098ee7440 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f062dffb640 1 -- 172.21.15.31:0/332926670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f06380b8740 con 0x558098ee7440 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f064f85a740 10 client.15285 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f064f85a740 10 client.15285 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f064f85a740 20 client.15285 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f064f85a740 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098efe520 0x558098eff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f064f85a740 4 client.15285 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 --> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558098e58440 con 0x558098efe520 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f0645d9b640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098efe520 0x558098eff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f0645d9b640 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098efe520 0x558098eff990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b87a21b95292768a server_cookie=e05ee572eadba2e1 in_seq=0 out_seq=0 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.843+0000 7f062dffb640 10 client.15285 ms_handle_connect on v2:172.21.15.31:6834/310214957 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f062dffb640 1 -- 172.21.15.31:0/332926670 <== mds.3 v2:172.21.15.31:6834/310214957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558098e58440 con 0x558098efe520 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f062dffb640 10 client.15285 handle_command_reply: tid=0 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 client.15285 shutdown 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 msgr2=0x558098ef1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 0x558098ef1840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0638032fd0 tx=0x7f06380319b0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 shutdown_connections 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098efe520 0x558098eff990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098eecfa0 0x558098ef8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7e30 0x558098ef1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 --2- 172.21.15.31:0/332926670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7440 0x558098ef1840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f062dffb640 0 client.15285 ms_handle_reset on v2:172.21.15.31:6834/310214957 2024-04-25T00:30:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 >> 172.21.15.31:0/332926670 conn(0x558098ee2ab0 msgr2=0x558098ee1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 shutdown_connections 2024-04-25T00:30:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 1 -- 172.21.15.31:0/332926670 wait complete. 2024-04-25T00:30:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.844+0000 7f064f85a740 20 client.15285 trim_cache size 0 max 16384 2024-04-25T00:30:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.862+0000 7f064f85a740 1 Processor -- start 2024-04-25T00:30:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.862+0000 7f064f85a740 1 -- start start 2024-04-25T00:30:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.862+0000 7f064f85a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.862+0000 7f064f85a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7e30 0x558098ef8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.862+0000 7f064f85a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 0x558098eef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.862+0000 7f064f85a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558098efdcd0 con 0x558098eecfa0 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f064f85a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558098efe3a0 con 0x558098ee7440 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f064f85a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558098efea70 con 0x558098ee7e30 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 0x558098eef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 0x558098eef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56070/0 (socket says 172.21.15.31:56070) 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 -- 172.21.15.31:0/947063836 learned_addr learned my addr 172.21.15.31:0/947063836 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f064659c640 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0645d9b640 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7e30 0x558098ef8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 -- 172.21.15.31:0/947063836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7e30 msgr2=0x558098ef8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7e30 0x558098ef8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 -- 172.21.15.31:0/947063836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 msgr2=0x558098ee88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 -- 172.21.15.31:0/947063836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558098edcc10 con 0x558098eecfa0 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f0646d9d640 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 0x558098eef390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f064000e5d0 tx=0x7f0640015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1262c565deff68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f062ffff640 1 -- 172.21.15.31:0/947063836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064001e790 con 0x558098eecfa0 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f062ffff640 1 -- 172.21.15.31:0/947063836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f064001ede0 con 0x558098eecfa0 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.863+0000 7f062ffff640 1 -- 172.21.15.31:0/947063836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064000f3c0 con 0x558098eecfa0 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- 172.21.15.31:0/947063836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 msgr2=0x558098eef390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 0x558098eef390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f064000e5d0 tx=0x7f0640015000 comp rx=0 tx=0).stop 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- 172.21.15.31:0/947063836 shutdown_connections 2024-04-25T00:30:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098eecfa0 0x558098eef390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7e30 0x558098ef8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- 172.21.15.31:0/947063836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098ee7440 0x558098ee88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- 172.21.15.31:0/947063836 >> 172.21.15.31:0/947063836 conn(0x558098ee2ab0 msgr2=0x558098ee2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- 172.21.15.31:0/947063836 shutdown_connections 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- 172.21.15.31:0/947063836 wait complete. 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- start start 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 Processor -- start 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7440 0x558098ef6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098ef7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098eecfa0 0x558098ef7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558098eff810 con 0x558098ee7e30 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558098eff140 con 0x558098eecfa0 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.864+0000 7f064f85a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558098effee0 con 0x558098ee7440 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f064f85a740 10 client.0 _ll_register_callbacks cb 0x7f06380bba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098ef7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098ef7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56074/0 (socket says 172.21.15.31:56074) 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 -- 172.21.15.31:0/2362125446 learned_addr learned my addr 172.21.15.31:0/2362125446 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0646d9d640 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098eecfa0 0x558098ef7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 -- 172.21.15.31:0/2362125446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7440 msgr2=0x558098ef6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7440 0x558098ef6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 -- 172.21.15.31:0/2362125446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098eecfa0 msgr2=0x558098ef7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098eecfa0 0x558098ef7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 -- 172.21.15.31:0/2362125446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558098ef8360 con 0x558098ee7e30 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f0645d9b640 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098ef7500 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f063c01c040 tx=0x7f063c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291ccb171e40271e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f062dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f062dffb640 1 -- 172.21.15.31:0/2362125446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c02c070 con 0x558098ee7e30 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f062dffb640 1 -- 172.21.15.31:0/2362125446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f063c0054c0 con 0x558098ee7e30 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f064001ede0 con 0x558098ee7e30 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f062dffb640 1 -- 172.21.15.31:0/2362125446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f063c028160 con 0x558098ee7e30 2024-04-25T00:30:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f064f85a740 10 client.15291 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558098f64840 con 0x558098ee7e30 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.865+0000 7f062dffb640 1 -- 172.21.15.31:0/2362125446 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f064001ede0 con 0x558098ee7e30 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f062dffb640 1 -- 172.21.15.31:0/2362125446 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f063c02f030 con 0x558098ee7e30 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 10 client.15291 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 10 client.15291 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 20 client.15291 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098f691b0 0x558098f67ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 4 client.15291 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 --> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] -- command(tid 0: {"prefix": "status"}) -- 0x558098e52790 con 0x558098f691b0 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064659c640 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098f691b0 0x558098f67ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064659c640 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098f691b0 0x558098f67ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d70eee9ace8e89f server_cookie=e4ee45741a6e2c7e in_seq=0 out_seq=0 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f062dffb640 10 client.15291 ms_handle_connect on v2:172.21.15.31:6834/310214957 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f062dffb640 1 -- 172.21.15.31:0/2362125446 <== mds.3 v2:172.21.15.31:6834/310214957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558098e52790 con 0x558098f691b0 2024-04-25T00:30:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f062dffb640 10 client.15291 handle_command_reply: tid=0 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 client.15291 shutdown 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 msgr2=0x558098ef7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098ef7500 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f063c01c040 tx=0x7f063c009eb0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 shutdown_connections 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:6834/310214957,v1:172.21.15.31:6835/310214957] conn(0x558098f691b0 0x558098f67ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558098eecfa0 0x558098ef7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f062dffb640 0 client.15291 ms_handle_reset on v2:172.21.15.31:6834/310214957 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558098ee7e30 0x558098ef7500 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 --2- 172.21.15.31:0/2362125446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558098ee7440 0x558098ef6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 >> 172.21.15.31:0/2362125446 conn(0x558098ee2ab0 msgr2=0x558098eec460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.866+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 shutdown_connections 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064f85a740 1 -- 172.21.15.31:0/2362125446 wait complete. 2024-04-25T00:30:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064f85a740 20 client.15291 trim_cache size 0 max 16384 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24434, 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:30:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:30:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 113.706630179, 2024-04-25T00:30:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:30:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-04-25T00:30:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:30:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 150.04028871700001 2024-04-25T00:30:21.877 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f06300a2b90 msgr2=0x7f06300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f06300a2b90 0x7f06300a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0640005fd0 tx=0x7f0640009b40 comp rx=0 tx=0).stop 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 msgr2=0x7f0648197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648197a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f06380314e0 tx=0x7f0638004270 comp rx=0 tx=0).stop 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 shutdown_connections 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0648102230 0x7f064819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0648101870 0x7f0648197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f06300a2b90 0x7f06300a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 --2- 172.21.15.31:0/4084296126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0648100680 0x7f0648197a00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 >> 172.21.15.31:0/4084296126 conn(0x7f06480fbbf0 msgr2=0x7f0648068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 shutdown_connections 2024-04-25T00:30:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.867+0000 7f064cb4f640 1 -- 172.21.15.31:0/4084296126 wait complete. 2024-04-25T00:30:21.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:30:21.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3543319132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:30:21.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2602443516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:30:21.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:21 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/516417450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:21.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3543319132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:30:21.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2602443516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:30:21.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:21 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/516417450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:21.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2353482513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 msgr2=0x7f12c4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2353482513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c4108ae0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009ca0 tx=0x7f12bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2353482513 shutdown_connections 2024-04-25T00:30:21.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2353482513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c4108ae0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2353482513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c4101890 0x7f12c4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2353482513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c41006a0 0x7f12c4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2353482513 >> 172.21.15.31:0/2353482513 conn(0x7f12c40fbc30 msgr2=0x7f12c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2353482513 shutdown_connections 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.982+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2353482513 wait complete. 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 Processor -- start 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 -- start start 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c41006a0 0x7f12c4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c4101890 0x7f12c4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410c1b0 con 0x7f12c4102250 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410c030 con 0x7f12c4101890 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12ca3dc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410c330 con 0x7f12c41006a0 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12c8979640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12c8979640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56090/0 (socket says 172.21.15.31:56090) 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12c8979640 1 -- 172.21.15.31:0/2633897759 learned_addr learned my addr 172.21.15.31:0/2633897759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12c37fe640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c4101890 0x7f12c4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.983+0000 7f12c3fff640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c41006a0 0x7f12c4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c8979640 1 -- 172.21.15.31:0/2633897759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c41006a0 msgr2=0x7f12c4197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c8979640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c41006a0 0x7f12c4197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c8979640 1 -- 172.21.15.31:0/2633897759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c4101890 msgr2=0x7f12c4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c8979640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c4101890 0x7f12c4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c8979640 1 -- 172.21.15.31:0/2633897759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c419f530 con 0x7f12c4102250 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c37fe640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c4101890 0x7f12c4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c8979640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c419eec0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009dc0 tx=0x7f12bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c17fa640 1 -- 172.21.15.31:0/2633897759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc032ec0 con 0x7f12c4102250 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c419f7c0 con 0x7f12c4102250 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c17fa640 1 -- 172.21.15.31:0/2633897759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12bc0044f0 con 0x7f12c4102250 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12c17fa640 1 -- 172.21.15.31:0/2633897759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc030d40 con 0x7f12c4102250 2024-04-25T00:30:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.984+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c419fd60 con 0x7f12c4102250 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.986+0000 7f12c17fa640 1 -- 172.21.15.31:0/2633897759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12bc004040 con 0x7f12c4102250 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.986+0000 7f12cd1c6740 1 Processor -- start 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 -- start start 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba76860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0ba7b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55df0b9d9f00 con 0x55df0ba76e30 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df0b982750 con 0x55df0ba7bfa0 2024-04-25T00:30:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df0b9d5ec0 con 0x55df0ba76440 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0ba7b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c17fa640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ac0a2d30 0x7f12ac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0ba7b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56106/0 (socket says 172.21.15.31:56106) 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 -- 172.21.15.31:0/3677644575 learned_addr learned my addr 172.21.15.31:0/3677644575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c37fe640 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba76860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c8979640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ac0a2d30 0x7f12ac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 -- 172.21.15.31:0/3677644575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 msgr2=0x55df0ba76860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba76860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 -- 172.21.15.31:0/3677644575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 msgr2=0x55df0ba7e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c37fe640 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c17fa640 1 -- 172.21.15.31:0/2633897759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f12bc0f07a0 con 0x7f12c4102250 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 -- 172.21.15.31:0/3677644575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0b6ee2d0 con 0x55df0ba76e30 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c3fff640 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0ba7b930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f12b400cc40 tx=0x7f12b400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392f3d0333e12b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.987+0000 7f12c8979640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ac0a2d30 0x7f12ac0a51f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0b9940 tx=0x7f12bc090030 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12aa7fc640 1 -- 172.21.15.31:0/3677644575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4014070 con 0x55df0ba76e30 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12aa7fc640 1 -- 172.21.15.31:0/3677644575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12b40054d0 con 0x55df0ba76e30 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12aa7fc640 1 -- 172.21.15.31:0/3677644575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4004cf0 con 0x55df0ba76e30 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 -- 172.21.15.31:0/3677644575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 msgr2=0x55df0ba7b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0ba7b930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f12b400cc40 tx=0x7f12b400e720 comp rx=0 tx=0).stop 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 -- 172.21.15.31:0/3677644575 shutdown_connections 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0ba7b930 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/3677644575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba76860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 -- 172.21.15.31:0/3677644575 >> 172.21.15.31:0/3677644575 conn(0x55df0ba71ab0 msgr2=0x55df0ba71eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 -- 172.21.15.31:0/3677644575 shutdown_connections 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.988+0000 7f12cd1c6740 1 -- 172.21.15.31:0/3677644575 wait complete. 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 -- start start 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 Processor -- start 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba80860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba80da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 0x55df0ba87de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba7ef50 con 0x55df0ba7bfa0 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df0b995cf0 con 0x55df0ba76e30 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba7f0d0 con 0x55df0ba76440 2024-04-25T00:30:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12cd1c6740 10 client.0 _ll_register_callbacks cb 0x55df0b9956e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 0x55df0ba87de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 0x55df0ba87de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56110/0 (socket says 172.21.15.31:56110) 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 -- 172.21.15.31:0/4271924263 learned_addr learned my addr 172.21.15.31:0/4271924263 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba80860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c3fff640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba80da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 -- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 msgr2=0x55df0ba80860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba80860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 -- 172.21.15.31:0/4271924263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 msgr2=0x55df0ba80da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba80da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.989+0000 7f12c37fe640 1 -- 172.21.15.31:0/4271924263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0b7b86b0 con 0x55df0ba7bfa0 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12c37fe640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 0x55df0ba87de0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f12b800bf90 tx=0x7f12b800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abee882cc2a4dfe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12a37fe640 1 -- 172.21.15.31:0/4271924263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8011c70 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12a37fe640 1 -- 172.21.15.31:0/4271924263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f12b8011e10 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b40054d0 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12a37fe640 1 -- 172.21.15.31:0/4271924263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f12b8004020 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 10 client.15303 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df0ba8ae90 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12a37fe640 1 -- 172.21.15.31:0/4271924263 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12b40054d0 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12a37fe640 1 -- 172.21.15.31:0/4271924263 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f12b8028070 con 0x55df0ba7bfa0 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 10 client.15303 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 20 client.15303 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0ba8d530 0x55df0ba8e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 4 client.15303 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.990+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 --> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0b9e7440 con 0x55df0ba8d530 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.991+0000 7f12c3fff640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0ba8d530 0x55df0ba8e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.991+0000 7f12c3fff640 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0ba8d530 0x55df0ba8e9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af76bb507a2f17f1 server_cookie=12cac39130b00fd7 in_seq=0 out_seq=0 2024-04-25T00:30:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.991+0000 7f12a37fe640 10 client.15303 ms_handle_connect on v2:172.21.15.169:6836/1684471092 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12a37fe640 1 -- 172.21.15.31:0/4271924263 <== mds.2 v2:172.21.15.169:6836/1684471092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df0b9e7440 con 0x55df0ba8d530 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12a37fe640 10 client.15303 handle_command_reply: tid=0 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 client.15303 shutdown 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 msgr2=0x55df0ba87de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 0x55df0ba87de0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f12b800bf90 tx=0x7f12b800bc60 comp rx=0 tx=0).stop 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 shutdown_connections 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0ba8d530 0x55df0ba8e9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba7bfa0 0x55df0ba87de0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba80da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12a37fe640 0 client.15303 ms_handle_reset on v2:172.21.15.169:6836/1684471092 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/4271924263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0ba80860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 >> 172.21.15.31:0/4271924263 conn(0x55df0ba71ab0 msgr2=0x55df0ba70ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 shutdown_connections 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.992+0000 7f12cd1c6740 1 -- 172.21.15.31:0/4271924263 wait complete. 2024-04-25T00:30:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:21.993+0000 7f12cd1c6740 20 client.15303 trim_cache size 0 max 16384 2024-04-25T00:30:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.010+0000 7f12cd1c6740 1 Processor -- start 2024-04-25T00:30:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 -- start start 2024-04-25T00:30:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 0x55df0ba80470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba809b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba8cce0 con 0x55df0ba76440 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba8d3b0 con 0x55df0ba76e30 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba8da80 con 0x55df0ba7bfa0 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 0x55df0ba80470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 0x55df0ba80470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56126/0 (socket says 172.21.15.31:56126) 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 -- 172.21.15.31:0/1868480213 learned_addr learned my addr 172.21.15.31:0/1868480213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 -- 172.21.15.31:0/1868480213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba7bfa0 msgr2=0x55df0ba7f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 -- 172.21.15.31:0/1868480213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 msgr2=0x55df0ba809b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 -- 172.21.15.31:0/1868480213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0ba71570 con 0x55df0ba76440 2024-04-25T00:30:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12c37fe640 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 0x55df0ba80470 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005bc0 tx=0x7f12b800b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971c15b78d3d9416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12a9ffb640 1 -- 172.21.15.31:0/1868480213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8011c40 con 0x55df0ba76440 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12a9ffb640 1 -- 172.21.15.31:0/1868480213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12b802e070 con 0x55df0ba76440 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.011+0000 7f12a9ffb640 1 -- 172.21.15.31:0/1868480213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b802a490 con 0x55df0ba76440 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 -- 172.21.15.31:0/1868480213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 msgr2=0x55df0ba80470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 0x55df0ba80470 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005bc0 tx=0x7f12b800b680 comp rx=0 tx=0).stop 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 -- 172.21.15.31:0/1868480213 shutdown_connections 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba7bfa0 0x55df0ba7f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba76e30 0x55df0ba809b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/1868480213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76440 0x55df0ba80470 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 -- 172.21.15.31:0/1868480213 >> 172.21.15.31:0/1868480213 conn(0x55df0ba71ab0 msgr2=0x55df0ba71e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 -- 172.21.15.31:0/1868480213 shutdown_connections 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 -- 172.21.15.31:0/1868480213 wait complete. 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 -- start start 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.012+0000 7f12cd1c6740 1 Processor -- start 2024-04-25T00:30:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0baef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0baefc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0baf6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba8e820 con 0x55df0ba76e30 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba8e150 con 0x55df0ba7bfa0 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df0ba8eef0 con 0x55df0ba76440 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 10 client.0 _ll_register_callbacks cb 0x7f12b8012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0baefc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0baefc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56142/0 (socket says 172.21.15.31:56142) 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 -- 172.21.15.31:0/53320909 learned_addr learned my addr 172.21.15.31:0/53320909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c37fe640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0baef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 -- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 msgr2=0x55df0baef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0baef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c37fe640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0baf6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 -- 172.21.15.31:0/53320909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 msgr2=0x55df0baf6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0baf6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 -- 172.21.15.31:0/53320909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0baf6d10 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c37fe640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0baf6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12c3fff640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0baefc70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12b400ec20 tx=0x7f12b400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aead724542853a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12a37fe640 1 -- 172.21.15.31:0/53320909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4014040 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12a37fe640 1 -- 172.21.15.31:0/53320909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f12b4026230 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.013+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b802e070 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12a37fe640 1 -- 172.21.15.31:0/53320909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f12b4026450 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 10 client.15309 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df0baf80a0 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12a37fe640 1 -- 172.21.15.31:0/53320909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12b802e070 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12a37fe640 1 -- 172.21.15.31:0/53320909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f12b4031c40 con 0x55df0ba76e30 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 10 client.15309 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 10 client.15309 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:30:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 20 client.15309 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0bafcaf0 0x55df0bafb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 4 client.15309 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 --> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] -- command(tid 0: {"prefix": "status"}) -- 0x55df0b9e1790 con 0x55df0bafcaf0 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.014+0000 7f12c37fe640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0bafcaf0 0x55df0bafb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12c37fe640 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0bafcaf0 0x55df0bafb800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1042f604773fbd22 server_cookie=9ba0d6d34a6ebed1 in_seq=0 out_seq=0 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12a37fe640 10 client.15309 ms_handle_connect on v2:172.21.15.169:6836/1684471092 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12a37fe640 1 -- 172.21.15.31:0/53320909 <== mds.2 v2:172.21.15.169:6836/1684471092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55df0b9e1790 con 0x55df0bafcaf0 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12a37fe640 10 client.15309 handle_command_reply: tid=0 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 client.15309 shutdown 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 msgr2=0x55df0baefc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0baefc70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12b400ec20 tx=0x7f12b400cb40 comp rx=0 tx=0).stop 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 shutdown_connections 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:6836/1684471092,v1:172.21.15.169:6837/1684471092] conn(0x55df0bafcaf0 0x55df0bafb800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df0ba7bfa0 0x55df0baf6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df0ba76e30 0x55df0baefc70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 --2- 172.21.15.31:0/53320909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df0ba76440 0x55df0baef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12a37fe640 0 client.15309 ms_handle_reset on v2:172.21.15.169:6836/1684471092 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 >> 172.21.15.31:0/53320909 conn(0x55df0ba71ab0 msgr2=0x55df0ba80470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 shutdown_connections 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 1 -- 172.21.15.31:0/53320909 wait complete. 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.015+0000 7f12cd1c6740 20 client.15309 trim_cache size 0 max 16384 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24499, 2024-04-25T00:30:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 114.582730954, 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 138.46887055100001 2024-04-25T00:30:22.028 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:30:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ac0a2d30 msgr2=0x7f12ac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ac0a2d30 0x7f12ac0a51f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0b9940 tx=0x7f12bc090030 comp rx=0 tx=0).stop 2024-04-25T00:30:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 msgr2=0x7f12c419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c419eec0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009dc0 tx=0x7f12bc003ba0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 shutdown_connections 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f12ac0a2d30 0x7f12ac0a51f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c4102250 0x7f12c419eec0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c4101890 0x7f12c4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 --2- 172.21.15.31:0/2633897759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c41006a0 0x7f12c4197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 >> 172.21.15.31:0/2633897759 conn(0x7f12c40fbc30 msgr2=0x7f12c40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 shutdown_connections 2024-04-25T00:30:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.016+0000 7f12ca3dc640 1 -- 172.21.15.31:0/2633897759 wait complete. 2024-04-25T00:30:22.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 -- 172.21.15.31:0/289943442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0103350 msgr2=0x7f1da0109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 --2- 172.21.15.31:0/289943442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0103350 0x7f1da0109be0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d9802a410 comp rx=0 tx=0).stop 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 -- 172.21.15.31:0/289943442 shutdown_connections 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 --2- 172.21.15.31:0/289943442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0103350 0x7f1da0109be0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 --2- 172.21.15.31:0/289943442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0102990 0x7f1da0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 --2- 172.21.15.31:0/289943442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da01017a0 0x7f1da0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 -- 172.21.15.31:0/289943442 >> 172.21.15.31:0/289943442 conn(0x7f1da00fcd10 msgr2=0x7f1da00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 -- 172.21.15.31:0/289943442 shutdown_connections 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.132+0000 7f1da514f640 1 -- 172.21.15.31:0/289943442 wait complete. 2024-04-25T00:30:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 Processor -- start 2024-04-25T00:30:22.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 -- start start 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 0x7f1da0197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0102990 0x7f1da0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0103350 0x7f1da019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d2b0 con 0x7f1da01017a0 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d130 con 0x7f1da0102990 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1da514f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d430 con 0x7f1da0103350 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 0x7f1da0197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 0x7f1da0197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56156/0 (socket says 172.21.15.31:56156) 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/713549599 learned_addr learned my addr 172.21.15.31:0/713549599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0102990 0x7f1da0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0103350 0x7f1da019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/713549599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0103350 msgr2=0x7f1da019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0103350 0x7f1da019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/713549599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0102990 msgr2=0x7f1da0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0102990 0x7f1da0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.133+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/713549599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019f5a0 con 0x7f1da01017a0 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.134+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 0x7f1da0197980 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009fd0 tx=0x7f1d9000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.134+0000 7f1d87fff640 1 -- 172.21.15.31:0/713549599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90011470 con 0x7f1da01017a0 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.134+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019f890 con 0x7f1da01017a0 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.134+0000 7f1d87fff640 1 -- 172.21.15.31:0/713549599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d90011ac0 con 0x7f1da01017a0 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.134+0000 7f1d87fff640 1 -- 172.21.15.31:0/713549599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001aab0 con 0x7f1da01017a0 2024-04-25T00:30:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.134+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da019fde0 con 0x7f1da01017a0 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.136+0000 7f1d87fff640 1 -- 172.21.15.31:0/713549599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d90011c70 con 0x7f1da01017a0 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.136+0000 7f1da7e5c740 1 Processor -- start 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d87fff640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d880a2b90 0x7f1d880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 -- start start 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d880a2b90 0x7f1d880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d87fff640 1 -- 172.21.15.31:0/713549599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f1d900cf5c0 con 0x7f1da01017a0 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475f0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 0x55e0475f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e0475f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e047553f00 con 0x55e0475f0e30 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0474fc750 con 0x55e0475f5fa0 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e04754fec0 con 0x55e0475f0440 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 0x55e0475f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 0x55e0475f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56168/0 (socket says 172.21.15.31:56168) 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3062425373 learned_addr learned my addr 172.21.15.31:0/3062425373 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3062425373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 msgr2=0x55e0475f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3062425373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 msgr2=0x55e0475f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e0475f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d880a2b90 0x7f1d880a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d98006010 tx=0x7f1d98009640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 -- 172.21.15.31:0/3062425373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0472682d0 con 0x55e0475f0e30 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 0x55e0475f5930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400bc30 tx=0x7f1d940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221a7240e41cae6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.137+0000 7f1d857fa640 1 -- 172.21.15.31:0/3062425373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9401c480 con 0x55e0475f0e30 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1d857fa640 1 -- 172.21.15.31:0/3062425373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d9401cad0 con 0x55e0475f0e30 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1d857fa640 1 -- 172.21.15.31:0/3062425373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94011af0 con 0x55e0475f0e30 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 -- 172.21.15.31:0/3062425373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 msgr2=0x55e0475f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 0x55e0475f5930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400bc30 tx=0x7f1d940107a0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 -- 172.21.15.31:0/3062425373 shutdown_connections 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e0475f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0e30 0x55e0475f5930 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/3062425373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 -- 172.21.15.31:0/3062425373 >> 172.21.15.31:0/3062425373 conn(0x55e0475ebab0 msgr2=0x55e0475ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 -- 172.21.15.31:0/3062425373 shutdown_connections 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.138+0000 7f1da7e5c740 1 -- 172.21.15.31:0/3062425373 wait complete. 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 -- start start 2024-04-25T00:30:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 Processor -- start 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 0x55e0475fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e047601db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0475f8f50 con 0x55e0475f0440 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e04750fcf0 con 0x55e0475f5fa0 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e0475f90d0 con 0x55e0475f0e30 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56184/0 (socket says 172.21.15.31:56184) 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/53195792 learned_addr learned my addr 172.21.15.31:0/53195792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1da7e5c740 10 client.0 _ll_register_callbacks cb 0x55e04750f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e047601db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/53195792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 msgr2=0x55e0475fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 0x55e0475fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 msgr2=0x55e047601db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e047601db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/53195792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0473326b0 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e047601db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475fa830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c770 tx=0x7f1d9009c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984604afd8a8218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d82ff5640 1 -- 172.21.15.31:0/53195792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001ae50 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.139+0000 7f1d82ff5640 1 -- 172.21.15.31:0/53195792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f1d900cf110 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9401cad0 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1d82ff5640 1 -- 172.21.15.31:0/53195792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d900a4a00 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 10 client.15321 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e047604e50 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1d82ff5640 1 -- 172.21.15.31:0/53195792 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d9401cad0 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1d82ff5640 1 -- 172.21.15.31:0/53195792 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1d90022030 con 0x55e0475f0440 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 10 client.15321 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 20 client.15321 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047607520 0x55e047608990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 4 client.15321 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 --> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e047561440 con 0x55e047607520 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.140+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047607520 0x55e047608990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.141+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047607520 0x55e047608990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcfa37952acbf7e server_cookie=e933c8ce68bd7889 in_seq=0 out_seq=0 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.141+0000 7f1d82ff5640 10 client.15321 ms_handle_connect on v2:172.21.15.40:6838/2724938586 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.141+0000 7f1d82ff5640 1 -- 172.21.15.31:0/53195792 <== mds.1 v2:172.21.15.40:6838/2724938586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e047561440 con 0x55e047607520 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.141+0000 7f1d82ff5640 10 client.15321 handle_command_reply: tid=0 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.141+0000 7f1da7e5c740 1 client.15321 shutdown 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 msgr2=0x55e0475fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475fa830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c770 tx=0x7f1d9009c550 comp rx=0 tx=0).stop 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 shutdown_connections 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047607520 0x55e047608990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e047601db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 0x55e0475fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1d82ff5640 0 client.15321 ms_handle_reset on v2:172.21.15.40:6838/2724938586 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/53195792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475fa830 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 >> 172.21.15.31:0/53195792 conn(0x55e0475ebab0 msgr2=0x55e0475eaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 shutdown_connections 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 1 -- 172.21.15.31:0/53195792 wait complete. 2024-04-25T00:30:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.142+0000 7f1da7e5c740 20 client.15321 trim_cache size 0 max 16384 2024-04-25T00:30:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 Processor -- start 2024-04-25T00:30:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 -- start start 2024-04-25T00:30:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 0x55e047601d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e0475f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e047606cd0 con 0x55e0475f0440 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0476073a0 con 0x55e0475f5fa0 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e047607a70 con 0x55e0475f0e30 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56190/0 (socket says 172.21.15.31:56190) 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/42838098 learned_addr learned my addr 172.21.15.31:0/42838098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/42838098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 msgr2=0x55e047601d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 0x55e047601d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/42838098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 msgr2=0x55e0475f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e0475f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 -- 172.21.15.31:0/42838098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0475eb570 con 0x55e0475f0440 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.160+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475e6690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1d9009ba90 tx=0x7f1d90098160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb9aaa92e0280e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1d84ff9640 1 -- 172.21.15.31:0/42838098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001ae50 con 0x55e0475f0440 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1d84ff9640 1 -- 172.21.15.31:0/42838098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d90081070 con 0x55e0475f0440 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1d84ff9640 1 -- 172.21.15.31:0/42838098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9009b440 con 0x55e0475f0440 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 -- 172.21.15.31:0/42838098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 msgr2=0x55e0475e6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475e6690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1d9009ba90 tx=0x7f1d90098160 comp rx=0 tx=0).stop 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 -- 172.21.15.31:0/42838098 shutdown_connections 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f5fa0 0x55e0475f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0e30 0x55e047601d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/42838098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f0440 0x55e0475e6690 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 -- 172.21.15.31:0/42838098 >> 172.21.15.31:0/42838098 conn(0x55e0475ebab0 msgr2=0x55e0475ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 -- 172.21.15.31:0/42838098 shutdown_connections 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 -- 172.21.15.31:0/42838098 wait complete. 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 -- start start 2024-04-25T00:30:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.161+0000 7f1da7e5c740 1 Processor -- start 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f0e30 0x55e0476004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 0x55e047600a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e047608810 con 0x55e0475f5fa0 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e047608140 con 0x55e0475f0e30 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e047608ee0 con 0x55e0475f0440 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 10 client.0 _ll_register_callbacks cb 0x7f1d9009b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 0x55e047600a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 0x55e047600a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56204/0 (socket says 172.21.15.31:56204) 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 -- 172.21.15.31:0/2187519659 learned_addr learned my addr 172.21.15.31:0/2187519659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 -- 172.21.15.31:0/2187519659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 msgr2=0x55e0475fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9e59c640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f0e30 0x55e0476004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 -- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f0e30 msgr2=0x55e0476004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f0e30 0x55e0476004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 -- 172.21.15.31:0/2187519659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0476012c0 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d9f59e640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 0x55e047600a90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1d98033000 tx=0x7f1d98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef57bacb13f5a752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d82ff5640 1 -- 172.21.15.31:0/2187519659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9804e070 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1d82ff5640 1 -- 172.21.15.31:0/2187519659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f1d98004450 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.162+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d90081070 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1d82ff5640 1 -- 172.21.15.31:0/2187519659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d9805a050 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 10 client.15327 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e04766d850 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1d82ff5640 1 -- 172.21.15.31:0/2187519659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d90081070 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1d82ff5640 1 -- 172.21.15.31:0/2187519659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1d9804a7b0 con 0x55e0475f5fa0 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 10 client.15327 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 20 client.15327 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047672450 0x55e047671160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 4 client.15327 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 --> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] -- command(tid 0: {"prefix": "status"}) -- 0x55e04755b790 con 0x55e047672450 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.163+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047672450 0x55e047671160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1d82ff5640 10 client.15327 ms_handle_connect on v2:172.21.15.40:6838/2724938586 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1d9ed9d640 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047672450 0x55e047671160 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=336bd4328ac1e624 server_cookie=37952544e32fd142 in_seq=0 out_seq=0 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1d82ff5640 1 -- 172.21.15.31:0/2187519659 <== mds.1 v2:172.21.15.40:6838/2724938586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e04755b790 con 0x55e047672450 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1d82ff5640 10 client.15327 handle_command_reply: tid=0 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 client.15327 shutdown 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 msgr2=0x55e047600a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 0x55e047600a90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1d98033000 tx=0x7f1d98009260 comp rx=0 tx=0).stop 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 shutdown_connections 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:6838/2724938586,v1:172.21.15.40:6839/2724938586] conn(0x55e047672450 0x55e047671160 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0475f5fa0 0x55e047600a90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0475f0e30 0x55e0476004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1d82ff5640 0 client.15327 ms_handle_reset on v2:172.21.15.40:6838/2724938586 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 --2- 172.21.15.31:0/2187519659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0475f0440 0x55e0475fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 >> 172.21.15.31:0/2187519659 conn(0x55e0475ebab0 msgr2=0x55e0475e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 shutdown_connections 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 1 -- 172.21.15.31:0/2187519659 wait complete. 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da7e5c740 20 client.15327 trim_cache size 0 max 16384 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:30:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24512, 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 115.740346954, 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 124.083989984 2024-04-25T00:30:22.172 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d880a2b90 msgr2=0x7f1d880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.164+0000 7f1da514f640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d880a2b90 0x7f1d880a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d98006010 tx=0x7f1d98009640 comp rx=0 tx=0).stop 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 msgr2=0x7f1da0197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 0x7f1da0197980 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009fd0 tx=0x7f1d9000c270 comp rx=0 tx=0).stop 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 shutdown_connections 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0103350 0x7f1da019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0102990 0x7f1da0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1d880a2b90 0x7f1d880a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 --2- 172.21.15.31:0/713549599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da01017a0 0x7f1da0197980 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 >> 172.21.15.31:0/713549599 conn(0x7f1da00fcd10 msgr2=0x7f1da0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 shutdown_connections 2024-04-25T00:30:22.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.165+0000 7f1da514f640 1 -- 172.21.15.31:0/713549599 wait complete. 2024-04-25T00:30:22.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 -- 172.21.15.31:0/3057232762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 msgr2=0x7ff950073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 --2- 172.21.15.31:0/3057232762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff950073b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009900 tx=0x7ff94c02a480 comp rx=0 tx=0).stop 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 -- 172.21.15.31:0/3057232762 shutdown_connections 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 --2- 172.21.15.31:0/3057232762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff95010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 --2- 172.21.15.31:0/3057232762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff950074120 0x7ff950074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 --2- 172.21.15.31:0/3057232762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff950073b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 -- 172.21.15.31:0/3057232762 >> 172.21.15.31:0/3057232762 conn(0x7ff9500fcd10 msgr2=0x7ff9500ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 -- 172.21.15.31:0/3057232762 shutdown_connections 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.278+0000 7ff9585a9640 1 -- 172.21.15.31:0/3057232762 wait complete. 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 Processor -- start 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 -- start start 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff950074120 0x7ff950199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff95019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff9501a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff95010f4b0 con 0x7ff950109650 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff95010f330 con 0x7ff950075700 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff9585a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff95010f630 con 0x7ff950074120 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff9501a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff9501a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56232/0 (socket says 172.21.15.31:56232) 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 -- 172.21.15.31:0/1909983489 learned_addr learned my addr 172.21.15.31:0/1909983489 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956345640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff950074120 0x7ff950199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff955b44640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff95019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 -- 172.21.15.31:0/1909983489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff950074120 msgr2=0x7ff950199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff950074120 0x7ff950199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 -- 172.21.15.31:0/1909983489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 msgr2=0x7ff95019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.279+0000 7ff956b46640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff95019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff956b46640 1 -- 172.21.15.31:0/1909983489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9501a17f0 con 0x7ff950109650 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff955b44640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff95019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff956b46640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff9501a1180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff94800eb10 tx=0x7ff94800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff9477fe640 1 -- 172.21.15.31:0/1909983489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948004b20 con 0x7ff950109650 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9501a1ae0 con 0x7ff950109650 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff9477fe640 1 -- 172.21.15.31:0/1909983489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff948004cc0 con 0x7ff950109650 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff9477fe640 1 -- 172.21.15.31:0/1909983489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948010690 con 0x7ff950109650 2024-04-25T00:30:22.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.280+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a1fe0 con 0x7ff950109650 2024-04-25T00:30:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff9477fe640 1 -- 172.21.15.31:0/1909983489 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff948005020 con 0x7ff950109650 2024-04-25T00:30:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 Processor -- start 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 -- start start 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594dd5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594dda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594ddd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559594d38f00 con 0x559594dd5440 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559594ce1750 con 0x559594dd5e30 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff95b397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559594d34ec0 con 0x559594ddafa0 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.283+0000 7ff955b44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594dd5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594dd5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56244/0 (socket says 172.21.15.31:56244) 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 -- 172.21.15.31:0/1576745252 learned_addr learned my addr 172.21.15.31:0/1576745252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff956345640 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594dda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff9477fe640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff93c0a2b90 0x7ff93c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594ddd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff956b46640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff93c0a2b90 0x7ff93c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 -- 172.21.15.31:0/1576745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 msgr2=0x559594ddd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594ddd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 -- 172.21.15.31:0/1576745252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 msgr2=0x559594dda930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594dda930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 -- 172.21.15.31:0/1576745252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559594a4d2d0 con 0x559594dd5440 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff956345640 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594dda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff9477fe640 1 -- 172.21.15.31:0/1909983489 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff9480cf690 con 0x7ff950109650 2024-04-25T00:30:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff955b44640 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594dd5860 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff940013590 tx=0x7ff9400097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2d67a5df2a9bad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff944bfe640 1 -- 172.21.15.31:0/1576745252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000f520 con 0x559594dd5440 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff956b46640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff93c0a2b90 0x7ff93c0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff948098890 tx=0x7ff9480c6a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff944bfe640 1 -- 172.21.15.31:0/1576745252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff94000fb70 con 0x559594dd5440 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.284+0000 7ff944bfe640 1 -- 172.21.15.31:0/1576745252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400119c0 con 0x559594dd5440 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 -- 172.21.15.31:0/1576745252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 msgr2=0x559594dd5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594dd5860 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff940013590 tx=0x7ff9400097d0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 -- 172.21.15.31:0/1576745252 shutdown_connections 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594ddd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594dda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 --2- 172.21.15.31:0/1576745252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594dd5860 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 -- 172.21.15.31:0/1576745252 >> 172.21.15.31:0/1576745252 conn(0x559594dd0ab0 msgr2=0x559594dd0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 -- 172.21.15.31:0/1576745252 shutdown_connections 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 -- 172.21.15.31:0/1576745252 wait complete. 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 -- start start 2024-04-25T00:30:22.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.285+0000 7ff95b397740 1 Processor -- start 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594ddf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594ddfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594de6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559594dddf50 con 0x559594dd5440 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559594cf4cf0 con 0x559594dd5e30 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559594dde0d0 con 0x559594ddafa0 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594ddf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 10 client.0 _ll_register_callbacks cb 0x559594cf46e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594ddf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56250/0 (socket says 172.21.15.31:56250) 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 -- 172.21.15.31:0/4177414284 learned_addr learned my addr 172.21.15.31:0/4177414284 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 -- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 msgr2=0x559594de6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594de6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 -- 172.21.15.31:0/4177414284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 msgr2=0x559594ddfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff956345640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594ddfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594ddfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 -- 172.21.15.31:0/4177414284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559594b176b0 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff956345640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594ddfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff955b44640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594ddf800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff940013560 tx=0x7ff94000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1ba339b0874cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff939ffb640 1 -- 172.21.15.31:0/4177414284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000f520 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff939ffb640 1 -- 172.21.15.31:0/4177414284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff94000f6c0 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.286+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94000fb70 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff939ffb640 1 -- 172.21.15.31:0/4177414284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff94000f8c0 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 10 client.15339 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559594de9d60 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff939ffb640 1 -- 172.21.15.31:0/4177414284 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff94000fb70 con 0x559594dd5440 2024-04-25T00:30:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff939ffb640 1 -- 172.21.15.31:0/4177414284 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9400037b0 con 0x559594dd5440 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 10 client.15339 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 20 client.15339 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594dec150 0x559594ded5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 4 client.15339 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559594d46440 con 0x559594dec150 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.287+0000 7ff956345640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594dec150 0x559594ded5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.288+0000 7ff939ffb640 10 client.15339 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.288+0000 7ff956345640 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594dec150 0x559594ded5c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfeb2b84d1525bc server_cookie=856f6ea717d7c5b0 in_seq=0 out_seq=0 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.288+0000 7ff939ffb640 1 -- 172.21.15.31:0/4177414284 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559594d46440 con 0x559594dec150 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff939ffb640 10 client.15339 handle_command_reply: tid=0 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 client.15339 shutdown 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 msgr2=0x559594ddf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594ddf800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff940013560 tx=0x7ff94000ff50 comp rx=0 tx=0).stop 2024-04-25T00:30:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 shutdown_connections 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594dec150 0x559594ded5c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594de6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5e30 0x559594ddfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 --2- 172.21.15.31:0/4177414284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5440 0x559594ddf800 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 >> 172.21.15.31:0/4177414284 conn(0x559594dd0ab0 msgr2=0x559594dcfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff939ffb640 0 client.15339 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 shutdown_connections 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 1 -- 172.21.15.31:0/4177414284 wait complete. 2024-04-25T00:30:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.289+0000 7ff95b397740 20 client.15339 trim_cache size 0 max 16384 2024-04-25T00:30:22.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 Processor -- start 2024-04-25T00:30:22.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 -- start start 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 0x559594ddf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594dd5e30 0x559594de6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 0x559594ddf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559594deb900 con 0x559594ddafa0 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559594debfd0 con 0x559594dd5440 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff95b397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559594dec6a0 con 0x559594dd5e30 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 0x559594ddf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 0x559594ddf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56264/0 (socket says 172.21.15.31:56264) 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 -- 172.21.15.31:0/4044665712 learned_addr learned my addr 172.21.15.31:0/4044665712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 -- 172.21.15.31:0/4044665712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594dd5e30 msgr2=0x559594de6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594dd5e30 0x559594de6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 -- 172.21.15.31:0/4044665712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 msgr2=0x559594ddf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 0x559594ddf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.307+0000 7ff955b44640 1 -- 172.21.15.31:0/4044665712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559594dd6210 con 0x559594ddafa0 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff955b44640 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 0x559594ddf950 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff940001eb0 tx=0x7ff9400134c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d6e8fec95d428c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff93bfff640 1 -- 172.21.15.31:0/4044665712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940003e10 con 0x559594ddafa0 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff93bfff640 1 -- 172.21.15.31:0/4044665712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff940026030 con 0x559594ddafa0 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff93bfff640 1 -- 172.21.15.31:0/4044665712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940009650 con 0x559594ddafa0 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 -- 172.21.15.31:0/4044665712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 msgr2=0x559594ddf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 0x559594ddf950 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff940001eb0 tx=0x7ff9400134c0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 -- 172.21.15.31:0/4044665712 shutdown_connections 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594ddafa0 0x559594ddf950 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594dd5e30 0x559594de6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 --2- 172.21.15.31:0/4044665712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 0x559594ddf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 -- 172.21.15.31:0/4044665712 >> 172.21.15.31:0/4044665712 conn(0x559594dd0ab0 msgr2=0x559594deb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 -- 172.21.15.31:0/4044665712 shutdown_connections 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.308+0000 7ff95b397740 1 -- 172.21.15.31:0/4044665712 wait complete. 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 -- start start 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 Processor -- start 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 0x559594de4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 0x559594e4e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594e50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559594ded440 con 0x559594dd5e30 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559594decd70 con 0x559594dd5440 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559594dedb10 con 0x559594ddafa0 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff95b397740 10 client.0 _ll_register_callbacks cb 0x7ff9400043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 0x559594e4e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 0x559594e4e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56278/0 (socket says 172.21.15.31:56278) 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 -- 172.21.15.31:0/853148696 learned_addr learned my addr 172.21.15.31:0/853148696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff955b44640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594e50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 -- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 msgr2=0x559594e50630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594e50630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 -- 172.21.15.31:0/853148696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 msgr2=0x559594de4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 0x559594de4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 -- 172.21.15.31:0/853148696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559594e50bf0 con 0x559594dd5e30 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff955b44640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594e50630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff956345640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 0x559594e4e0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009900 tx=0x7ff94c031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2dce3695c7333d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.309+0000 7ff939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff939ffb640 1 -- 172.21.15.31:0/853148696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c035040 con 0x559594dd5e30 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff939ffb640 1 -- 172.21.15.31:0/853148696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff94c045dc0 con 0x559594dd5e30 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff940026030 con 0x559594dd5e30 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff939ffb640 1 -- 172.21.15.31:0/853148696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff94c004040 con 0x559594dd5e30 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 10 client.15345 fetch_fsmap learned FSMap version 25 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559594e51f60 con 0x559594dd5e30 2024-04-25T00:30:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff939ffb640 1 -- 172.21.15.31:0/853148696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff940026030 con 0x559594dd5e30 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff939ffb640 1 -- 172.21.15.31:0/853148696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff94c0036f0 con 0x559594dd5e30 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 10 client.15345 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 10 client.15345 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 20 client.15345 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594e56bb0 0x559594e52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 4 client.15345 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.310+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "status"}) -- 0x559594d40790 con 0x559594e56bb0 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff955b44640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594e56bb0 0x559594e52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff939ffb640 10 client.15345 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff955b44640 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594e56bb0 0x559594e52ff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2a3d27f7839044 server_cookie=6a83e9044a601ca8 in_seq=0 out_seq=0 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff939ffb640 1 -- 172.21.15.31:0/853148696 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559594d40790 con 0x559594e56bb0 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff939ffb640 10 client.15345 handle_command_reply: tid=0 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 client.15345 shutdown 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 msgr2=0x559594e4e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 0x559594e4e0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009900 tx=0x7ff94c031730 comp rx=0 tx=0).stop 2024-04-25T00:30:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 shutdown_connections 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x559594e56bb0 0x559594e52ff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559594ddafa0 0x559594e50630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff939ffb640 0 client.15345 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559594dd5e30 0x559594e4e0e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 --2- 172.21.15.31:0/853148696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559594dd5440 0x559594de4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 >> 172.21.15.31:0/853148696 conn(0x559594dd0ab0 msgr2=0x559594dda3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 shutdown_connections 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.311+0000 7ff95b397740 1 -- 172.21.15.31:0/853148696 wait complete. 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff95b397740 20 client.15345 trim_cache size 0 max 16384 2024-04-25T00:30:22.318 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "6c091b24-0299-11ef-bc93-c7b262605968", 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24517, 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 117.90571334400001, 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 132.951898959 2024-04-25T00:30:22.319 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff93c0a2b90 msgr2=0x7ff93c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff93c0a2b90 0x7ff93c0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff948098890 tx=0x7ff9480c6a50 comp rx=0 tx=0).stop 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 msgr2=0x7ff9501a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff9501a1180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff94800eb10 tx=0x7ff94800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 shutdown_connections 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff93c0a2b90 0x7ff93c0a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff950109650 0x7ff9501a1180 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950075700 0x7ff95019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 --2- 172.21.15.31:0/1909983489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff950074120 0x7ff950199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 >> 172.21.15.31:0/1909983489 conn(0x7ff9500fcd10 msgr2=0x7ff9501055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 shutdown_connections 2024-04-25T00:30:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:22.312+0000 7ff9585a9640 1 -- 172.21.15.31:0/1909983489 wait complete. 2024-04-25T00:30:22.323 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:30:22.331 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:30:22.332 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:30:22.332 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:30:22.332 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T00:30:22.354 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T00:30:22.354 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T00:30:22.354 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-25T00:30:22.354 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:30:22.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:22.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:22.355 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T00:30:22.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:22.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-25T00:30:22.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:22 smithi031 ceph-mon[29167]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:30:22.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:22 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3959279747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:22.437 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-04-25T00:30:22.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:22.438 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T00:30:22.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:30:22.724 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:30:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:22 smithi169 ceph-mon[33112]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:30:22.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:22 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3959279747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:22 smithi040 ceph-mon[32945]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:30:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:22 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3959279747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:30:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:24 smithi031 ceph-mon[29167]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 75 B/s, 0 objects/s recovering 2024-04-25T00:30:24.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 -- 172.21.15.31:0/2083758013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91141024e0 msgr2=0x7f91141028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:24.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 --2- 172.21.15.31:0/2083758013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91141024e0 0x7f91141028c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9104009990 tx=0x7f9104009ca0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 -- 172.21.15.31:0/2083758013 shutdown_connections 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 --2- 172.21.15.31:0/2083758013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91141024e0 0x7f91141028c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 --2- 172.21.15.31:0/2083758013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9114107570 0x7f911410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 --2- 172.21.15.31:0/2083758013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9114102e90 0x7f9114106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 -- 172.21.15.31:0/2083758013 >> 172.21.15.31:0/2083758013 conn(0x7f9114079390 msgr2=0x7f91140797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 -- 172.21.15.31:0/2083758013 shutdown_connections 2024-04-25T00:30:24.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 -- 172.21.15.31:0/2083758013 wait complete. 2024-04-25T00:30:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 Processor -- start 2024-04-25T00:30:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.590+0000 7f911a743700 1 -- start start 2024-04-25T00:30:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f911a743700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91141024e0 0x7f9114074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f911a743700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 0x7f9114074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f911a743700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9114107570 0x7f9114074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f911a743700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9114112400 con 0x7f9114102e90 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f911a743700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9114112280 con 0x7f9114107570 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f911a743700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9114112580 con 0x7f91141024e0 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 0x7f9114074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 0x7f9114074a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57054/0 (socket says 172.21.15.31:57054) 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 -- 172.21.15.31:0/1771411099 learned_addr learned my addr 172.21.15.31:0/1771411099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f9113fff700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91141024e0 0x7f9114074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f9118ce0700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9114107570 0x7f9114074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 -- 172.21.15.31:0/1771411099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91141024e0 msgr2=0x7f9114074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91141024e0 0x7f9114074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 -- 172.21.15.31:0/1771411099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9114107570 msgr2=0x7f9114074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9114107570 0x7f9114074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.591+0000 7f91137fe700 1 -- 172.21.15.31:0/1771411099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a6de0 con 0x7f9114102e90 2024-04-25T00:30:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.592+0000 7f91137fe700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 0x7f9114074a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f910800e9f0 tx=0x7f910800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.592+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910800cbd0 con 0x7f9114102e90 2024-04-25T00:30:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.592+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a70d0 con 0x7f9114102e90 2024-04-25T00:30:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.592+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f910800cd70 con 0x7f9114102e90 2024-04-25T00:30:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.592+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108010430 con 0x7f9114102e90 2024-04-25T00:30:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.592+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a7610 con 0x7f9114102e90 2024-04-25T00:30:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.593+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911410a790 con 0x7f9114102e90 2024-04-25T00:30:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.594+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9108004d70 con 0x7f9114102e90 2024-04-25T00:30:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.594+0000 7f91117fa700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.595+0000 7f9113fff700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.595+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f91080cf2d0 con 0x7f9114102e90 2024-04-25T00:30:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.595+0000 7f9113fff700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f910400b2e0 tx=0x7f9104005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.599+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9108005700 con 0x7f9114102e90 2024-04-25T00:30:24.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:24 smithi169 ceph-mon[33112]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 75 B/s, 0 objects/s recovering 2024-04-25T00:30:24.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.854+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f911410a170 con 0x7f9114102e90 2024-04-25T00:30:24.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.855+0000 7f91117fa700 1 -- 172.21.15.31:0/1771411099 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f910809c200 con 0x7f9114102e90 2024-04-25T00:30:24.855 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f90fc0a2c20 msgr2=0x7f90fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f910400b2e0 tx=0x7f9104005960 comp rx=0 tx=0).stop 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 msgr2=0x7f9114074a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 0x7f9114074a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f910800e9f0 tx=0x7f910800ed00 comp rx=0 tx=0).stop 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 shutdown_connections 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9114107570 0x7f9114074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91141024e0 0x7f9114074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 --2- 172.21.15.31:0/1771411099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9114102e90 0x7f9114074a70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 >> 172.21.15.31:0/1771411099 conn(0x7f9114079390 msgr2=0x7f91141086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 shutdown_connections 2024-04-25T00:30:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:24.857+0000 7f911a743700 1 -- 172.21.15.31:0/1771411099 wait complete. 2024-04-25T00:30:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:24 smithi040 ceph-mon[32945]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 75 B/s, 0 objects/s recovering 2024-04-25T00:30:25.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:25 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1771411099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:30:25.588 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:30:25.588 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:30:25.588 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-04-25T00:30:25.589 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:30:25.589 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:25.589 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:30:25.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:30:25.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:30:25.589 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:30:25.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:25.589 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-04-25T00:30:25.606 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:30:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:30:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-04-25T00:30:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42066sec preferred_lft 42066sec 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-04-25T00:30:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-04-25T00:30:25.608 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:30:25.608 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:30:25.609 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T00:30:25.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:30:25.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-04-25T00:30:25.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-04-25T00:30:25.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:30:25.609 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T00:30:25.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:30:25.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:30:25.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:25.798 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:30:25.824 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-04-25T00:30:25.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:25.826 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-04-25T00:30:25.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:25 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1771411099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:30:25.895 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-04-25T00:30:25.895 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:30:25.895 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T00:30:25.895 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T00:30:25.895 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:30:25.896 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:30:25.896 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T00:30:25.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:30:25.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:30:25.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:30:25.896 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T00:30:25.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:25 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1771411099' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:30:25.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:30:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:30:26.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:26.172 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T00:30:26.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:26.188 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-04-25T00:30:26.243 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:30:26.243 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T00:30:26.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:26.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:30:26.243 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T00:30:26.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:30:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:30:26.361 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:30:26.361 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:30:26.362 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T00:30:26.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:30:26.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:30:26.507 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:30:26.507 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T00:30:26.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-04-25T00:30:26.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:30:26.507 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T00:30:26.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:30:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:30:26.583 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-25T00:30:26.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:30:26.584 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:26.598 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:30:26.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:30:26.599 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:26.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-25T00:30:26.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:26 smithi031 ceph-mon[29167]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 67 B/s, 0 objects/s recovering 2024-04-25T00:30:26.718 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:30:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-25T00:30:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:30:26.776 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/7fe5cac086d2dc8c6e56bb05f97fb081b8d1301714483ee2c5cf27ba81da03cc/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/1b6a1f5973f199892dbeedeeb4f006958cdbc0a88b2e0aace9514c3df8f511cd/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b1f3054909dc87487461ca456cfeef3f4e1f6915327ab559cc7c2be0ba655639/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e58d2996f5d6ded1f2f842a15b8054ac1ba07c041fdd938d05561f93b741193f/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/484cc52f0a211757991b5902c8e8b7c1042f4f8ee56d5cb7bed0be3288522fb3/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/26656c38a1609729874112f3e1ade1d60ef980917f12e9b10f8a6cd0791ad1f6/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/966755fd03bfbbfce5cd964df8cdc42e1f6bbdf07a54dfd83bfb50aa9a868b28/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/f0a4e299df6b378dd269d15f98b3e51e0715df01d50e106a9dcbc957637a8af9/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/607eaf91a88cdb62df4175dd8144a6c19d73671464510bf81cf8880d41a97e1b/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/72fb88b50aca1f3771435ecb37964713eb04a05fa0d6d3a7258ed643f85de46a/merged 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:26.777 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:26.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:26.778 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-25T00:30:26.796 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:26 smithi040 ceph-mon[32945]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 67 B/s, 0 objects/s recovering 2024-04-25T00:30:26.830 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:30:26.830 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/bf15116c-d217-4ec3-a264-717aa9c7473f --client_fs=cephfs) 2024-04-25T00:30:26.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-25T00:30:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:26 smithi169 ceph-mon[33112]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 67 B/s, 0 objects/s recovering 2024-04-25T00:30:26.896 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:30:26.953 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-25T00:30:26.953 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-25T00:30:26.953 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-25T00:30:26.953 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-25T00:30:26.953 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-25T00:30:26.953 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-25T00:30:26.954 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:30:26.955 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:30:26.955 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:30:26.955 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:30:26.957 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-25T00:30:26.957 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-25T00:30:26.957 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:30:26.957 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/7fe5cac086d2dc8c6e56bb05f97fb081b8d1301714483ee2c5cf27ba81da03cc/merged 2024-04-25T00:30:26.957 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/1b6a1f5973f199892dbeedeeb4f006958cdbc0a88b2e0aace9514c3df8f511cd/merged 2024-04-25T00:30:26.957 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b1f3054909dc87487461ca456cfeef3f4e1f6915327ab559cc7c2be0ba655639/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e58d2996f5d6ded1f2f842a15b8054ac1ba07c041fdd938d05561f93b741193f/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/484cc52f0a211757991b5902c8e8b7c1042f4f8ee56d5cb7bed0be3288522fb3/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/26656c38a1609729874112f3e1ade1d60ef980917f12e9b10f8a6cd0791ad1f6/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/966755fd03bfbbfce5cd964df8cdc42e1f6bbdf07a54dfd83bfb50aa9a868b28/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/f0a4e299df6b378dd269d15f98b3e51e0715df01d50e106a9dcbc957637a8af9/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/607eaf91a88cdb62df4175dd8144a6c19d73671464510bf81cf8880d41a97e1b/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/72fb88b50aca1f3771435ecb37964713eb04a05fa0d6d3a7258ed643f85de46a/merged 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:26.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:26.959 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-25T00:30:26.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-04-25T00:30:26.971+0000 7fec44488480 -1 init, newargv = 0x5589c70e2240 newargc=15 2024-04-25T00:30:26.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[78687]: starting ceph client 2024-04-25T00:30:26.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[78687]: starting fuse 2024-04-25T00:30:28.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-25T00:30:28.119 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-25T00:30:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:30:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/7fe5cac086d2dc8c6e56bb05f97fb081b8d1301714483ee2c5cf27ba81da03cc/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/1b6a1f5973f199892dbeedeeb4f006958cdbc0a88b2e0aace9514c3df8f511cd/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b1f3054909dc87487461ca456cfeef3f4e1f6915327ab559cc7c2be0ba655639/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e58d2996f5d6ded1f2f842a15b8054ac1ba07c041fdd938d05561f93b741193f/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/484cc52f0a211757991b5902c8e8b7c1042f4f8ee56d5cb7bed0be3288522fb3/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/26656c38a1609729874112f3e1ade1d60ef980917f12e9b10f8a6cd0791ad1f6/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/966755fd03bfbbfce5cd964df8cdc42e1f6bbdf07a54dfd83bfb50aa9a868b28/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/f0a4e299df6b378dd269d15f98b3e51e0715df01d50e106a9dcbc957637a8af9/merged 2024-04-25T00:30:28.180 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/607eaf91a88cdb62df4175dd8144a6c19d73671464510bf81cf8880d41a97e1b/merged 2024-04-25T00:30:28.181 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/72fb88b50aca1f3771435ecb37964713eb04a05fa0d6d3a7258ed643f85de46a/merged 2024-04-25T00:30:28.181 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-25T00:30:28.181 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:28.181 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:30:28.181 INFO:teuthology.orchestra.run.smithi031.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:28.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:28.181 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-25T00:30:28.236 INFO:teuthology.orchestra.run.smithi031.stdout:115 2024-04-25T00:30:28.236 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-04-25T00:30:28.237 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-04-25T00:30:28.238 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-04-25T00:30:28.363 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.78687.asok 2024-04-25T00:30:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:30:28.375 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.78687.asok 2024-04-25T00:30:28.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:28.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.78687.asok status 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-04-25T00:30:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "78687", 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/volumes/qa/sv_0/bf15116c-d217-4ec3-a264-717aa9c7473f" 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 15357, 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 15357 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-25T00:30:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2861232470 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2861232470 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.15357 192.168.0.1:0/2861232470", 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2861232470", 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 5, 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 25, 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 91, 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:30:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-04-25T00:30:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-04-25T00:30:28.516 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:30:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:28.522 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-04-25T00:30:28.522 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:28.537 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-04-25T00:30:28.538 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:28.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:28.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:28.605 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:30:28.615 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:30:28.615 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:30:28.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T00:30:28.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:28 smithi031 ceph-mon[29167]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s; 56 B/s, 0 objects/s recovering 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 -- 172.21.15.31:0/3051236787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c0103380 msgr2=0x7f92c0109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 --2- 172.21.15.31:0/3051236787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c0103380 0x7f92c0109c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009ca0 tx=0x7f92b802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 -- 172.21.15.31:0/3051236787 shutdown_connections 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 --2- 172.21.15.31:0/3051236787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c0103380 0x7f92c0109c10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 --2- 172.21.15.31:0/3051236787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c01029c0 0x7f92c0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 --2- 172.21.15.31:0/3051236787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92c01017d0 0x7f92c0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 -- 172.21.15.31:0/3051236787 >> 172.21.15.31:0/3051236787 conn(0x7f92c00fcd60 msgr2=0x7f92c00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 -- 172.21.15.31:0/3051236787 shutdown_connections 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.747+0000 7f92c522a640 1 -- 172.21.15.31:0/3051236787 wait complete. 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 Processor -- start 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 -- start start 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 0x7f92c01978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92c01029c0 0x7f92c0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c0103380 0x7f92c019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f92c010d300 con 0x7f92c01017d0 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92c010d180 con 0x7f92c0103380 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92c522a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92c010d480 con 0x7f92c01029c0 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92bed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 0x7f92c01978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92bed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 0x7f92c01978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57062/0 (socket says 172.21.15.31:57062) 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92bed9d640 1 -- 172.21.15.31:0/3198956923 learned_addr learned my addr 172.21.15.31:0/3198956923 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92b7fff640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92c01029c0 0x7f92c0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.748+0000 7f92bf59e640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c0103380 0x7f92c019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bed9d640 1 -- 172.21.15.31:0/3198956923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92c01029c0 msgr2=0x7f92c0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bed9d640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92c01029c0 0x7f92c0197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bed9d640 1 -- 172.21.15.31:0/3198956923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c0103380 msgr2=0x7f92c019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bed9d640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c0103380 0x7f92c019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bed9d640 1 -- 172.21.15.31:0/3198956923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c019f510 con 0x7f92c01017d0 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bed9d640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 0x7f92c01978f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f92b000d950 tx=0x7f92b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0017070 con 0x7f92c01017d0 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92b0009c30 con 0x7f92c01017d0 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c019f800 con 0x7f92c01017d0 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b000f620 con 0x7f92c01017d0 2024-04-25T00:30:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.749+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c019fd50 con 0x7f92c01017d0 2024-04-25T00:30:28.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.750+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c0108af0 con 0x7f92c01017d0 2024-04-25T00:30:28.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.751+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92b001a070 con 0x7f92c01017d0 2024-04-25T00:30:28.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.752+0000 7f92bcd99640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f92ac0a2cb0 0x7f92ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:28.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.752+0000 7f92b7fff640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f92ac0a2cb0 0x7f92ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:28.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.752+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f92b00ceaf0 con 0x7f92c01017d0 2024-04-25T00:30:28.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.752+0000 7f92b7fff640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f92ac0a2cb0 0x7f92ac0a5170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92c0198e00 tx=0x7f92a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.756+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f92b0009de0 con 0x7f92c01017d0 2024-04-25T00:30:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:28 smithi169 ceph-mon[33112]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s; 56 B/s, 0 objects/s recovering 2024-04-25T00:30:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:28 smithi040 ceph-mon[32945]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s; 56 B/s, 0 objects/s recovering 2024-04-25T00:30:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:28.981+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f92c002d170 con 0x7f92c01017d0 2024-04-25T00:30:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.437+0000 7f92bcd99640 1 -- 172.21.15.31:0/3198956923 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f92b009b9c0 con 0x7f92c01017d0 2024-04-25T00:30:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f92ac0a2cb0 msgr2=0x7f92ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f92ac0a2cb0 0x7f92ac0a5170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92c0198e00 tx=0x7f92a800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 msgr2=0x7f92c01978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 0x7f92c01978f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f92b000d950 tx=0x7f92b000de10 comp rx=0 tx=0).stop 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 shutdown_connections 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c0103380 0x7f92c019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92c01029c0 0x7f92c0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f92ac0a2cb0 0x7f92ac0a5170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 --2- 172.21.15.31:0/3198956923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92c01017d0 0x7f92c01978f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 >> 172.21.15.31:0/3198956923 conn(0x7f92c00fcd60 msgr2=0x7f92c0105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 shutdown_connections 2024-04-25T00:30:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.440+0000 7f92c522a640 1 -- 172.21.15.31:0/3198956923 wait complete. 2024-04-25T00:30:29.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T00:30:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 -- 172.21.15.31:0/758686699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 msgr2=0x7f1414101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 --2- 172.21.15.31:0/758686699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414101bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1410009900 tx=0x7f141002a410 comp rx=0 tx=0).stop 2024-04-25T00:30:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 -- 172.21.15.31:0/758686699 shutdown_connections 2024-04-25T00:30:29.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 --2- 172.21.15.31:0/758686699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1414103380 0x7f1414109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 --2- 172.21.15.31:0/758686699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14141029c0 0x7f1414102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 --2- 172.21.15.31:0/758686699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414101bf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 -- 172.21.15.31:0/758686699 >> 172.21.15.31:0/758686699 conn(0x7f14140fcd60 msgr2=0x7f14140ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 -- 172.21.15.31:0/758686699 shutdown_connections 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.603+0000 7f141c0fc640 1 -- 172.21.15.31:0/758686699 wait complete. 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 Processor -- start 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 -- start start 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14141029c0 0x7f1414197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1414103380 0x7f141419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f141410d350 con 0x7f14141017d0 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f141410d1d0 con 0x7f1414103380 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141c0fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f141410d4d0 con 0x7f14141029c0 2024-04-25T00:30:29.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57074/0 (socket says 172.21.15.31:57074) 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 -- 172.21.15.31:0/2286010696 learned_addr learned my addr 172.21.15.31:0/2286010696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419697640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14141029c0 0x7f1414197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f141a699640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1414103380 0x7f141419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 -- 172.21.15.31:0/2286010696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14141029c0 msgr2=0x7f1414197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14141029c0 0x7f1414197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 -- 172.21.15.31:0/2286010696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1414103380 msgr2=0x7f141419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1414103380 0x7f141419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 -- 172.21.15.31:0/2286010696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141419f520 con 0x7f14141017d0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.604+0000 7f1419e98640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414197900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1410009230 tx=0x7f1410004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.605+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141003f030 con 0x7f14141017d0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.605+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1410003cb0 con 0x7f14141017d0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.605+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141419f7b0 con 0x7f14141017d0 2024-04-25T00:30:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.605+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410035070 con 0x7f14141017d0 2024-04-25T00:30:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.605+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141419fd70 con 0x7f14141017d0 2024-04-25T00:30:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.606+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1414108a90 con 0x7f14141017d0 2024-04-25T00:30:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.607+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1410042030 con 0x7f14141017d0 2024-04-25T00:30:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.607+0000 7f140affd640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14000a2cb0 0x7f14000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:29.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.608+0000 7f1419697640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14000a2cb0 0x7f14000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:29.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.608+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f14100ef5a0 con 0x7f14141017d0 2024-04-25T00:30:29.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.608+0000 7f1419697640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14000a2cb0 0x7f14000a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1414198e10 tx=0x7f140400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:29.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.612+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14100bc4d0 con 0x7f14141017d0 2024-04-25T00:30:29.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:29 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3198956923' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:30:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.824+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 --> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1414061ce0 con 0x7f14000a2cb0 2024-04-25T00:30:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.842+0000 7f140affd640 1 -- 172.21.15.31:0/2286010696 <== mgr.14150 v2:172.21.15.31:6800/2153348461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1414061ce0 con 0x7f14000a2cb0 2024-04-25T00:30:29.843 INFO:teuthology.orchestra.run.smithi031.stdout:{} 2024-04-25T00:30:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.843+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14000a2cb0 msgr2=0x7f14000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14000a2cb0 0x7f14000a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1414198e10 tx=0x7f140400a2d0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 msgr2=0x7f1414197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414197900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1410009230 tx=0x7f1410004340 comp rx=0 tx=0).stop 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 shutdown_connections 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1414103380 0x7f141419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14141029c0 0x7f1414197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14000a2cb0 0x7f14000a5170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 --2- 172.21.15.31:0/2286010696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141017d0 0x7f1414197900 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 >> 172.21.15.31:0/2286010696 conn(0x7f14140fcd60 msgr2=0x7f1414105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 shutdown_connections 2024-04-25T00:30:29.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:30:29.844+0000 7f141c0fc640 1 -- 172.21.15.31:0/2286010696 wait complete. 2024-04-25T00:30:29.855 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T00:30:29.866 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T00:30:29.908 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:30:29.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T00:30:29.909 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:29 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3198956923' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:30:29.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:29 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3198956923' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:30:29.912 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T00:30:29.924 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:30:29.925 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T00:30:29.925 INFO:tasks.workunit:timeout=3h 2024-04-25T00:30:29.925 INFO:tasks.workunit:cleanup=True 2024-04-25T00:30:29.925 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 73h/115d Inode: 1 Links: 2 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 00:28:28.876246042 +0000 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 00:28:28.876246042 +0000 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 00:30:28.602377545 +0000 2024-04-25T00:30:29.941 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-04-25T00:30:29.941 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T00:30:29.942 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T00:30:30.013 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:30:30.069 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T00:30:30.274 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:30:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:30:29.828+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:30:30.274 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:30:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:30:29.828+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:30:30.274 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:30:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:30:29.828+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:30:30.274 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:30:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:30:29.828+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:30:30.274 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:30:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:30:29.828+0000 7fea17b87700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:30:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:30 smithi031 ceph-mon[29167]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-04-25T00:30:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:30 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3198956923' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T00:30:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:30 smithi031 ceph-mon[29167]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:30:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:30 smithi031 ceph-mon[29167]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:30 smithi169 ceph-mon[33112]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-04-25T00:30:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:30 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3198956923' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T00:30:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:30 smithi169 ceph-mon[33112]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:30:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:30 smithi169 ceph-mon[33112]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:30 smithi040 ceph-mon[32945]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-04-25T00:30:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:30 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3198956923' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T00:30:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:30 smithi040 ceph-mon[32945]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:30:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:30 smithi040 ceph-mon[32945]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:30:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:32 smithi031 ceph-mon[29167]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-25T00:30:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:32 smithi031 ceph-mon[29167]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:30:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:32 smithi169 ceph-mon[33112]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-25T00:30:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:32 smithi169 ceph-mon[33112]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:30:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:32 smithi040 ceph-mon[32945]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-25T00:30:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:32 smithi040 ceph-mon[32945]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:30:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:34 smithi031 ceph-mon[29167]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T00:30:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:34 smithi169 ceph-mon[33112]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T00:30:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:34 smithi040 ceph-mon[32945]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T00:30:36.563 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:36 smithi031 ceph-mon[29167]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T00:30:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:36 smithi169 ceph-mon[33112]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T00:30:36.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:36 smithi040 ceph-mon[32945]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T00:30:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:38 smithi031 ceph-mon[29167]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T00:30:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:38 smithi169 ceph-mon[33112]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T00:30:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:38 smithi040 ceph-mon[32945]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T00:30:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:40 smithi031 ceph-mon[29167]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T00:30:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:40 smithi169 ceph-mon[33112]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T00:30:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:40 smithi040 ceph-mon[32945]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T00:30:42.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:42 smithi031 ceph-mon[29167]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T00:30:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:42 smithi169 ceph-mon[33112]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T00:30:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:42 smithi040 ceph-mon[32945]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 -- 172.21.15.31:0/2537346205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941017d0 msgr2=0x7fa694101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 --2- 172.21.15.31:0/2537346205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941017d0 0x7fa694101bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa684009900 tx=0x7fa68402a410 comp rx=0 tx=0).stop 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 -- 172.21.15.31:0/2537346205 shutdown_connections 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 --2- 172.21.15.31:0/2537346205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa694103380 0x7fa694109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 --2- 172.21.15.31:0/2537346205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6941029c0 0x7fa694102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 --2- 172.21.15.31:0/2537346205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941017d0 0x7fa694101bf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 -- 172.21.15.31:0/2537346205 >> 172.21.15.31:0/2537346205 conn(0x7fa6940fcd60 msgr2=0x7fa6940ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 -- 172.21.15.31:0/2537346205 shutdown_connections 2024-04-25T00:30:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 -- 172.21.15.31:0/2537346205 wait complete. 2024-04-25T00:30:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.052+0000 7fa699f90640 1 Processor -- start 2024-04-25T00:30:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 -- start start 2024-04-25T00:30:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6941017d0 0x7fa6940754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 0x7fa694073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa694103380 0x7fa694074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410d300 con 0x7fa6941029c0 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410d180 con 0x7fa694103380 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa699f90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410d480 con 0x7fa6941017d0 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 0x7fa694073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 0x7fa694073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56076/0 (socket says 172.21.15.31:56076) 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 -- 172.21.15.31:0/2610777252 learned_addr learned my addr 172.21.15.31:0/2610777252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa6937fe640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6941017d0 0x7fa6940754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa693fff640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa694103380 0x7fa694074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 -- 172.21.15.31:0/2610777252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6941017d0 msgr2=0x7fa6940754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6941017d0 0x7fa6940754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 -- 172.21.15.31:0/2610777252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa694103380 msgr2=0x7fa694074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa694103380 0x7fa694074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa692ffd640 1 -- 172.21.15.31:0/2610777252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694074900 con 0x7fa6941029c0 2024-04-25T00:30:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.053+0000 7fa693fff640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa694103380 0x7fa694074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.054+0000 7fa692ffd640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 0x7fa694073b30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa68800c920 tx=0x7fa68800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.054+0000 7fa690ff9640 1 -- 172.21.15.31:0/2610777252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688007c70 con 0x7fa6941029c0 2024-04-25T00:30:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.054+0000 7fa690ff9640 1 -- 172.21.15.31:0/2610777252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa688007e10 con 0x7fa6941029c0 2024-04-25T00:30:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.054+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6941a1a10 con 0x7fa6941029c0 2024-04-25T00:30:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.054+0000 7fa690ff9640 1 -- 172.21.15.31:0/2610777252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688018820 con 0x7fa6941029c0 2024-04-25T00:30:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.054+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941a1f60 con 0x7fa6941029c0 2024-04-25T00:30:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.056+0000 7fa690ff9640 1 -- 172.21.15.31:0/2610777252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa6880189c0 con 0x7fa6941029c0 2024-04-25T00:30:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 Processor -- start 2024-04-25T00:30:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 -- start start 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf63270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf689b0 0x55977cf6ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55977cec6cb0 con 0x55977cf62e50 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55977ce6f5b0 con 0x55977cf689b0 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa69cd79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55977cec2c70 con 0x55977cf63840 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf63270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa690ff9640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa67c0a2cd0 0x7fa67c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf63270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56092/0 (socket says 172.21.15.31:56092) 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 -- 172.21.15.31:0/2336864013 learned_addr learned my addr 172.21.15.31:0/2336864013 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa6937fe640 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa692ffd640 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf689b0 0x55977cf6ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa67c0a2cd0 0x7fa67c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 -- 172.21.15.31:0/2336864013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 msgr2=0x55977cf68340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf68340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa690ff9640 1 -- 172.21.15.31:0/2610777252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa6880cf3d0 con 0x7fa6941029c0 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.057+0000 7fa693fff640 1 -- 172.21.15.31:0/2336864013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf689b0 msgr2=0x55977cf6ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa693fff640 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf689b0 0x55977cf6ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa693fff640 1 -- 172.21.15.31:0/2336864013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977cbdb210 con 0x55977cf62e50 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa692ffd640 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf689b0 0x55977cf6ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa693fff640 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf63270 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0129e0 tx=0x7fa68c012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecda4fae2e1f90e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa679ffb640 1 -- 172.21.15.31:0/2336864013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c022070 con 0x55977cf62e50 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa679ffb640 1 -- 172.21.15.31:0/2336864013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa68c003c30 con 0x55977cf62e50 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.058+0000 7fa679ffb640 1 -- 172.21.15.31:0/2336864013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c005220 con 0x55977cf62e50 2024-04-25T00:30:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 -- 172.21.15.31:0/2336864013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 msgr2=0x55977cf63270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf63270 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0129e0 tx=0x7fa68c012ea0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa693fff640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa67c0a2cd0 0x7fa67c0a5190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa68c020000 tx=0x7fa68c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 -- 172.21.15.31:0/2336864013 shutdown_connections 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf689b0 0x55977cf6ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf68340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 --2- 172.21.15.31:0/2336864013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf63270 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 -- 172.21.15.31:0/2336864013 >> 172.21.15.31:0/2336864013 conn(0x55977cf5e4c0 msgr2=0x55977cf5e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 -- 172.21.15.31:0/2336864013 shutdown_connections 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 -- 172.21.15.31:0/2336864013 wait complete. 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 -- start start 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.059+0000 7fa69cd79740 1 Processor -- start 2024-04-25T00:30:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 0x55977cf6d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55977cf6b960 con 0x55977cf63840 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55977ce82b50 con 0x55977cf62e50 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55977cf6bae0 con 0x55977cf689b0 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 10 client.0 _ll_register_callbacks cb 0x55977cebdca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 0x55977cf6d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 0x55977cf6d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56098/0 (socket says 172.21.15.31:56098) 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 -- 172.21.15.31:0/4155834810 learned_addr learned my addr 172.21.15.31:0/4155834810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa693fff640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 -- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 msgr2=0x55977cf74820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf74820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 -- 172.21.15.31:0/4155834810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 msgr2=0x55977cf6d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf6d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 -- 172.21.15.31:0/4155834810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977cca7620 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 0x55977cf6d7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa68809b7a0 tx=0x7fa688066b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46b7103c1e68de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa672ffd640 1 -- 172.21.15.31:0/4155834810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688066b80 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa672ffd640 1 -- 172.21.15.31:0/4155834810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa68809bd00 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.060+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa68c003c30 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa672ffd640 1 -- 172.21.15.31:0/4155834810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa68809c1a0 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 10 client.15387 fetch_fsmap learned FSMap version 26 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55977cf77c60 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa672ffd640 1 -- 172.21.15.31:0/4155834810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa68c003c30 con 0x55977cf63840 2024-04-25T00:30:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa672ffd640 1 -- 172.21.15.31:0/4155834810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6880a4280 con 0x55977cf63840 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 20 client.15387 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cf792e0 0x55977cf7b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 4 client.15387 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55977ced41f0 con 0x55977cf792e0 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.061+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cf792e0 0x55977cf7b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.062+0000 7fa672ffd640 10 client.15387 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.062+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cf792e0 0x55977cf7b750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ad4631db054a12 server_cookie=98d1eb2f9809b287 in_seq=0 out_seq=0 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa672ffd640 1 -- 172.21.15.31:0/4155834810 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55977ced41f0 con 0x55977cf792e0 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa672ffd640 10 client.15387 handle_command_reply: tid=0 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 client.15387 shutdown 2024-04-25T00:30:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 msgr2=0x55977cf6d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 0x55977cf6d7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa68809b7a0 tx=0x7fa688066b10 comp rx=0 tx=0).stop 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 shutdown_connections 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cf792e0 0x55977cf7b750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf74820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf63840 0x55977cf6d7e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa672ffd640 0 client.15387 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4155834810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 >> 172.21.15.31:0/4155834810 conn(0x55977cf5e4c0 msgr2=0x55977cf5d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 shutdown_connections 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 1 -- 172.21.15.31:0/4155834810 wait complete. 2024-04-25T00:30:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.063+0000 7fa69cd79740 20 client.15387 trim_cache size 0 max 16384 2024-04-25T00:30:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.078+0000 7fa69cd79740 1 Processor -- start 2024-04-25T00:30:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.078+0000 7fa69cd79740 1 -- start start 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf5cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 0x55977cf5d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa69cd79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941aadb0 con 0x55977cf689b0 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa69cd79740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941ac8f0 con 0x55977cf62e50 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa69cd79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941ab480 con 0x55977cf63840 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 0x55977cf5d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 0x55977cf5d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56108/0 (socket says 172.21.15.31:56108) 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 -- 172.21.15.31:0/1142720791 learned_addr learned my addr 172.21.15.31:0/1142720791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa692ffd640 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa6937fe640 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf5cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 -- 172.21.15.31:0/1142720791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 msgr2=0x55977cf747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 -- 172.21.15.31:0/1142720791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 msgr2=0x55977cf5cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf5cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 -- 172.21.15.31:0/1142720791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977cf58850 con 0x55977cf689b0 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa693fff640 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 0x55977cf5d0f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa68c012660 tx=0x7fa68c012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f8f69fe22d8c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa6797fa640 1 -- 172.21.15.31:0/1142720791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c037070 con 0x55977cf689b0 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa6797fa640 1 -- 172.21.15.31:0/1142720791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa68c005200 con 0x55977cf689b0 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.079+0000 7fa6797fa640 1 -- 172.21.15.31:0/1142720791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c022050 con 0x55977cf689b0 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 -- 172.21.15.31:0/1142720791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 msgr2=0x55977cf5d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 0x55977cf5d0f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa68c012660 tx=0x7fa68c012690 comp rx=0 tx=0).stop 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 -- 172.21.15.31:0/1142720791 shutdown_connections 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf689b0 0x55977cf5d0f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf63840 0x55977cf747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 --2- 172.21.15.31:0/1142720791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf62e50 0x55977cf5cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 -- 172.21.15.31:0/1142720791 >> 172.21.15.31:0/1142720791 conn(0x55977cf5e4c0 msgr2=0x55977cf640d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 -- 172.21.15.31:0/1142720791 shutdown_connections 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 -- 172.21.15.31:0/1142720791 wait complete. 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.080+0000 7fa69cd79740 1 -- start start 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 Processor -- start 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf71ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf63840 0x55977cf720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941ac220 con 0x55977cf62e50 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941abb50 con 0x55977cf63840 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941a8d90 con 0x55977cf689b0 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa69cd79740 10 client.0 _ll_register_callbacks cb 0x7fa68800fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf71ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf71ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56116/0 (socket says 172.21.15.31:56116) 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 -- 172.21.15.31:0/4045953082 learned_addr learned my addr 172.21.15.31:0/4045953082 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa693fff640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf63840 0x55977cf720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 -- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 msgr2=0x55977cf730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 -- 172.21.15.31:0/4045953082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf63840 msgr2=0x55977cf720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf63840 0x55977cf720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 -- 172.21.15.31:0/4045953082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977cf736e0 con 0x55977cf62e50 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa693fff640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf63840 0x55977cf720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.081+0000 7fa6937fe640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf71ba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa684009900 tx=0x7fa684032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ccee3f41286861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa672ffd640 1 -- 172.21.15.31:0/4045953082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684035040 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa672ffd640 1 -- 172.21.15.31:0/4045953082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa684004450 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55977cece540 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa672ffd640 1 -- 172.21.15.31:0/4045953082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa684047420 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 10 client.15393 fetch_fsmap learned FSMap version 26 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55977cf74a60 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa672ffd640 1 -- 172.21.15.31:0/4045953082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55977cece540 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa672ffd640 1 -- 172.21.15.31:0/4045953082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa684059070 con 0x55977cf62e50 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 10 client.15393 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 20 client.15393 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cfe0090 0x55977cfe24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 4 client.15393 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.082+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55977ce4c8e0 con 0x55977cfe0090 2024-04-25T00:30:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.083+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cfe0090 0x55977cfe24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.083+0000 7fa672ffd640 10 client.15393 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.083+0000 7fa692ffd640 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cfe0090 0x55977cfe24b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810269f206ebc5d4 server_cookie=48e45d1aeda90c2c in_seq=0 out_seq=0 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa672ffd640 1 -- 172.21.15.31:0/4045953082 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55977ce4c8e0 con 0x55977cfe0090 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa672ffd640 10 client.15393 handle_command_reply: tid=0 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 client.15393 shutdown 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 msgr2=0x55977cf71ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf71ba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa684009900 tx=0x7fa684032900 comp rx=0 tx=0).stop 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 shutdown_connections 2024-04-25T00:30:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55977cfe0090 0x55977cfe24b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55977cf689b0 0x55977cf730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55977cf63840 0x55977cf720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa672ffd640 0 client.15393 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 --2- 172.21.15.31:0/4045953082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55977cf62e50 0x55977cf71ba0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 >> 172.21.15.31:0/4045953082 conn(0x55977cf5e4c0 msgr2=0x55977cf61240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 shutdown_connections 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 1 -- 172.21.15.31:0/4045953082 wait complete. 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.096+0000 7fa69cd79740 20 client.15393 trim_cache size 0 max 16384 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:30:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:30:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:30:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:30:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa67c0a2cd0 msgr2=0x7fa67c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa67c0a2cd0 0x7fa67c0a5190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa68c020000 tx=0x7fa68c01f000 comp rx=0 tx=0).stop 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 msgr2=0x7fa694073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 0x7fa694073b30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa68800c920 tx=0x7fa68800cde0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 shutdown_connections 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa694103380 0x7fa694074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa67c0a2cd0 0x7fa67c0a5190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6941029c0 0x7fa694073b30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 --2- 172.21.15.31:0/2610777252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6941017d0 0x7fa6940754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 >> 172.21.15.31:0/2610777252 conn(0x7fa6940fcd60 msgr2=0x7fa694105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 shutdown_connections 2024-04-25T00:30:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:43.097+0000 7fa699f90640 1 -- 172.21.15.31:0/2610777252 wait complete. 2024-04-25T00:30:44.717 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:44 smithi169 ceph-mon[33112]: pgmap v366: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-25T00:30:44.718 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:44.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:44 smithi031 ceph-mon[29167]: pgmap v366: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-25T00:30:44.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:44.815 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:44 smithi040 ceph-mon[32945]: pgmap v366: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-25T00:30:44.815 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: pgmap v367: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T00:30:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:46.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: pgmap v367: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: pgmap v367: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:30:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:48 smithi031 ceph-mon[29167]: pgmap v368: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T00:30:48.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:48 smithi169 ceph-mon[33112]: pgmap v368: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T00:30:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:48 smithi040 ceph-mon[32945]: pgmap v368: 97 pgs: 97 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T00:30:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:50 smithi031 ceph-mon[29167]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 16 op/s 2024-04-25T00:30:50.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:50 smithi169 ceph-mon[33112]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 16 op/s 2024-04-25T00:30:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:50 smithi040 ceph-mon[32945]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 16 op/s 2024-04-25T00:30:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:52 smithi031 ceph-mon[29167]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T00:30:52.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:52 smithi169 ceph-mon[33112]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T00:30:52.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:52 smithi040 ceph-mon[32945]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T00:30:54.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:54 smithi169 ceph-mon[33112]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-04-25T00:30:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:54 smithi031 ceph-mon[29167]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-04-25T00:30:54.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:54 smithi040 ceph-mon[32945]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 -- 172.21.15.31:0/3001056117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b8102990 msgr2=0x7f51b8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 --2- 172.21.15.31:0/3001056117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b8102990 0x7f51b8102e10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009900 tx=0x7f51a002a410 comp rx=0 tx=0).stop 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 -- 172.21.15.31:0/3001056117 shutdown_connections 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 --2- 172.21.15.31:0/3001056117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8103350 0x7f51b8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 --2- 172.21.15.31:0/3001056117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b8102990 0x7f51b8102e10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 --2- 172.21.15.31:0/3001056117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b81017a0 0x7f51b8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 -- 172.21.15.31:0/3001056117 >> 172.21.15.31:0/3001056117 conn(0x7f51b80fcd30 msgr2=0x7f51b80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 -- 172.21.15.31:0/3001056117 shutdown_connections 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.199+0000 7f51bede7640 1 -- 172.21.15.31:0/3001056117 wait complete. 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 Processor -- start 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 -- start start 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 0x7f51b8197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8102990 0x7f51b8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b8103350 0x7f51b819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51b810d2d0 con 0x7f51b81017a0 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51b810d150 con 0x7f51b8102990 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bede7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51b810d450 con 0x7f51b8103350 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bcb83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 0x7f51b8197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bcb83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 0x7f51b8197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48558/0 (socket says 172.21.15.31:48558) 2024-04-25T00:30:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bcb83640 1 -- 172.21.15.31:0/2822309697 learned_addr learned my addr 172.21.15.31:0/2822309697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51bd384640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b8103350 0x7f51b819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.200+0000 7f51b7fff640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8102990 0x7f51b8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bcb83640 1 -- 172.21.15.31:0/2822309697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b8103350 msgr2=0x7f51b819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bcb83640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b8103350 0x7f51b819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bcb83640 1 -- 172.21.15.31:0/2822309697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8102990 msgr2=0x7f51b8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bcb83640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8102990 0x7f51b8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bcb83640 1 -- 172.21.15.31:0/2822309697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b819f540 con 0x7f51b81017a0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bcb83640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 0x7f51b8197920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9c0 tx=0x7f51a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51b5ffb640 1 -- 172.21.15.31:0/2822309697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a800cd80 con 0x7f51b81017a0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51b5ffb640 1 -- 172.21.15.31:0/2822309697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51a80035f0 con 0x7f51b81017a0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b819f830 con 0x7f51b81017a0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51b5ffb640 1 -- 172.21.15.31:0/2822309697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8010690 con 0x7f51b81017a0 2024-04-25T00:30:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.201+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b819fd00 con 0x7f51b81017a0 2024-04-25T00:30:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51b5ffb640 1 -- 172.21.15.31:0/2822309697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f51a80037a0 con 0x7f51b81017a0 2024-04-25T00:30:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 Processor -- start 2024-04-25T00:30:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 -- start start 2024-04-25T00:30:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a565762680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a565767750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a56576a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51b5ffb640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f51a40a2cd0 0x7f51a40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a5656c1cb0 con 0x55a565762c50 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a56566a5b0 con 0x55a565762260 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.204+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a5656bdc70 con 0x55a565767dc0 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a565767750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bd384640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f51a40a2cd0 0x7f51a40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a565767750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48568/0 (socket says 172.21.15.31:48568) 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 -- 172.21.15.31:0/901857997 learned_addr learned my addr 172.21.15.31:0/901857997 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bd384640 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a565762680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51b7fff640 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a56576a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51b5ffb640 1 -- 172.21.15.31:0/2822309697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f51a80cf730 con 0x7f51b81017a0 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 -- 172.21.15.31:0/901857997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 msgr2=0x55a56576a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a56576a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 -- 172.21.15.31:0/901857997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 msgr2=0x55a565762680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a565762680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 -- 172.21.15.31:0/901857997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5653d6210 con 0x55a565762c50 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bd384640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f51a40a2cd0 0x7f51a40a5190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005fd0 tx=0x7f51b0009de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51bcb83640 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a565767750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c6a0 tx=0x7f51a806eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38de1a2c1798d259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51aeffd640 1 -- 172.21.15.31:0/901857997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a809be00 con 0x55a565762c50 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51aeffd640 1 -- 172.21.15.31:0/901857997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51a80a01b0 con 0x55a565762c50 2024-04-25T00:30:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.205+0000 7f51aeffd640 1 -- 172.21.15.31:0/901857997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80abc70 con 0x55a565762c50 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 -- 172.21.15.31:0/901857997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 msgr2=0x55a565767750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a565767750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c6a0 tx=0x7f51a806eec0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 -- 172.21.15.31:0/901857997 shutdown_connections 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a56576a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a565767750 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/901857997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a565762680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 -- 172.21.15.31:0/901857997 >> 172.21.15.31:0/901857997 conn(0x55a5657594c0 msgr2=0x55a5657598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 -- 172.21.15.31:0/901857997 shutdown_connections 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 -- 172.21.15.31:0/901857997 wait complete. 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.206+0000 7f51c1bd1740 1 -- start start 2024-04-25T00:30:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 Processor -- start 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a56576c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a56576cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a565773bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a56576ad10 con 0x55a565762c50 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a56567db50 con 0x55a565762260 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a56576ae90 con 0x55a565767dc0 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51c1bd1740 10 client.0 _ll_register_callbacks cb 0x55a5656b8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a56576cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a56576cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48570/0 (socket says 172.21.15.31:48570) 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51b7fff640 1 -- 172.21.15.31:0/3954159682 learned_addr learned my addr 172.21.15.31:0/3954159682 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51bd384640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a565773bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51bcb83640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a56576c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51b7fff640 1 -- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 msgr2=0x55a565773bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.207+0000 7f51b7fff640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a565773bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51b7fff640 1 -- 172.21.15.31:0/3954159682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 msgr2=0x55a56576c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51b7fff640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a56576c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51b7fff640 1 -- 172.21.15.31:0/3954159682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5654a2620 con 0x55a565762c50 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51bd384640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a565773bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51b7fff640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a56576cb70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009900 tx=0x7f51a0031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50e02ad0196db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f5197fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f5197fff640 1 -- 172.21.15.31:0/3954159682 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0035070 con 0x55a565762c50 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51a80a01b0 con 0x55a565762c50 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f5197fff640 1 -- 172.21.15.31:0/3954159682 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f51a0004040 con 0x55a565762c50 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f5197fff640 1 -- 172.21.15.31:0/3954159682 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51a0004240 con 0x55a565762c50 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51c1bd1740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a565776fe0 con 0x55a565762c50 2024-04-25T00:30:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.208+0000 7f5197fff640 1 -- 172.21.15.31:0/3954159682 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51a80a01b0 con 0x55a565762c50 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f5197fff640 1 -- 172.21.15.31:0/3954159682 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51a004e070 con 0x55a565762c50 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51c1bd1740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51c1bd1740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51c1bd1740 20 client.15405 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657786d0 0x55a56577ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51c1bd1740 4 client.15405 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5656cf1f0 con 0x55a5657786d0 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51bcb83640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657786d0 0x55a56577ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f51bcb83640 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657786d0 0x55a56577ab50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6efd9a17db2a8d08 server_cookie=1ef14855e9c34c40 in_seq=0 out_seq=0 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.209+0000 7f5197fff640 10 client.15405 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f5197fff640 1 -- 172.21.15.31:0/3954159682 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5656cf1f0 con 0x55a5657786d0 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f5197fff640 10 client.15405 handle_command_reply: tid=0 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 client.15405 shutdown 2024-04-25T00:30:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 msgr2=0x55a56576cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a56576cb70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009900 tx=0x7f51a0031ef0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 shutdown_connections 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657786d0 0x55a56577ab50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565767dc0 0x55a565773bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565762c50 0x55a56576cb70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3954159682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762260 0x55a56576c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 >> 172.21.15.31:0/3954159682 conn(0x55a5657594c0 msgr2=0x55a5657585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f5197fff640 0 client.15405 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 shutdown_connections 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3954159682 wait complete. 2024-04-25T00:30:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.210+0000 7f51c1bd1740 20 client.15405 trim_cache size 0 max 16384 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.225+0000 7f51c1bd1740 1 Processor -- start 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.225+0000 7f51c1bd1740 1 -- start start 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565758440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565773b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a56577f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81a69a0 con 0x55a565767dc0 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81a84e0 con 0x55a565762c50 2024-04-25T00:30:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81a7070 con 0x55a565762260 2024-04-25T00:30:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a56577f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bcb83640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565758440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a56577f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48586/0 (socket says 172.21.15.31:48586) 2024-04-25T00:30:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 -- 172.21.15.31:0/36527596 learned_addr learned my addr 172.21.15.31:0/36527596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51b7fff640 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565773b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 -- 172.21.15.31:0/36527596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 msgr2=0x55a565758440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565758440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 -- 172.21.15.31:0/36527596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 msgr2=0x55a565773b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565773b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 -- 172.21.15.31:0/36527596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a565763030 con 0x55a565767dc0 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51b7fff640 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565773b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51bd384640 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a56577f450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009b00 tx=0x7f51b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f48eb51fb93b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.226+0000 7f51ae7fc640 1 -- 172.21.15.31:0/36527596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0003c70 con 0x55a565767dc0 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51ae7fc640 1 -- 172.21.15.31:0/36527596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b0003e10 con 0x55a565767dc0 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51ae7fc640 1 -- 172.21.15.31:0/36527596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0005020 con 0x55a565767dc0 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 -- 172.21.15.31:0/36527596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 msgr2=0x55a56577f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a56577f450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009b00 tx=0x7f51b001f000 comp rx=0 tx=0).stop 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 -- 172.21.15.31:0/36527596 shutdown_connections 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a56577f450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565773b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/36527596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565758440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 -- 172.21.15.31:0/36527596 >> 172.21.15.31:0/36527596 conn(0x55a5657594c0 msgr2=0x55a565757d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 -- 172.21.15.31:0/36527596 shutdown_connections 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.227+0000 7f51c1bd1740 1 -- 172.21.15.31:0/36527596 wait complete. 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 -- start start 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 Processor -- start 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565771ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565772010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a565773020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81a7e10 con 0x55a565767dc0 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81a7740 con 0x55a565762c50 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81a4980 con 0x55a565762260 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51c1bd1740 10 client.0 _ll_register_callbacks cb 0x7f51a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bd384640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a565773020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bd384640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a565773020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48588/0 (socket says 172.21.15.31:48588) 2024-04-25T00:30:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bd384640 1 -- 172.21.15.31:0/3468879014 learned_addr learned my addr 172.21.15.31:0/3468879014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bcb83640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565771ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51b7fff640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565772010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bd384640 1 -- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 msgr2=0x55a565771ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bd384640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565771ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.228+0000 7f51bd384640 1 -- 172.21.15.31:0/3468879014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 msgr2=0x55a565772010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51bd384640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565772010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51bd384640 1 -- 172.21.15.31:0/3468879014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a565773610 con 0x55a565767dc0 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51b7fff640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565772010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51bd384640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a565773020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f51b0010800 tx=0x7f51b00047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0b456f7dcae4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f5197fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f5197fff640 1 -- 172.21.15.31:0/3468879014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0004030 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b0003e10 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f5197fff640 1 -- 172.21.15.31:0/3468879014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f51b00041d0 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f5197fff640 1 -- 172.21.15.31:0/3468879014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51b003a050 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51c1bd1740 10 client.15411 fetch_fsmap learned FSMap version 26 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5657df500 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f5197fff640 1 -- 172.21.15.31:0/3468879014 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b0003e10 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.229+0000 7f5197fff640 1 -- 172.21.15.31:0/3468879014 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51b001f070 con 0x55a565767dc0 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51c1bd1740 10 client.15411 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51c1bd1740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:30:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51c1bd1740 20 client.15411 populate_metadata read hostname 'smithi031' 2024-04-25T00:30:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657dfe80 0x55a5657e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51c1bd1740 4 client.15411 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5656c9540 con 0x55a5657dfe80 2024-04-25T00:30:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51bcb83640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657dfe80 0x55a5657e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f5197fff640 10 client.15411 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.230+0000 7f51bcb83640 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657dfe80 0x55a5657e22a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0213c236b644e9e server_cookie=d4574c4b0d49d8fa in_seq=0 out_seq=0 2024-04-25T00:30:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f5197fff640 1 -- 172.21.15.31:0/3468879014 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5656c9540 con 0x55a5657dfe80 2024-04-25T00:30:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f5197fff640 10 client.15411 handle_command_reply: tid=0 2024-04-25T00:30:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 client.15411 shutdown 2024-04-25T00:30:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 msgr2=0x55a565773020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a565773020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f51b0010800 tx=0x7f51b00047f0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 shutdown_connections 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a5657dfe80 0x55a5657e22a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a565767dc0 0x55a565773020 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a565762c50 0x55a565772010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f5197fff640 0 client.15411 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 --2- 172.21.15.31:0/3468879014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a565762260 0x55a565771ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 >> 172.21.15.31:0/3468879014 conn(0x55a5657594c0 msgr2=0x55a5657585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 shutdown_connections 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 1 -- 172.21.15.31:0/3468879014 wait complete. 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.236+0000 7f51c1bd1740 20 client.15411 trim_cache size 0 max 16384 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:30:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:30:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:30:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:30:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f51a40a2cd0 msgr2=0x7f51a40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f51a40a2cd0 0x7f51a40a5190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005fd0 tx=0x7f51b0009de0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 msgr2=0x7f51b8197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 0x7f51b8197920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9c0 tx=0x7f51a800ee80 comp rx=0 tx=0).stop 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 shutdown_connections 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b8103350 0x7f51b819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8102990 0x7f51b8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f51a40a2cd0 0x7f51a40a5190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 --2- 172.21.15.31:0/2822309697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51b81017a0 0x7f51b8197920 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 >> 172.21.15.31:0/2822309697 conn(0x7f51b80fcd30 msgr2=0x7f51b8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 shutdown_connections 2024-04-25T00:30:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:30:56.237+0000 7f51bede7640 1 -- 172.21.15.31:0/2822309697 wait complete. 2024-04-25T00:30:56.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:56 smithi040 ceph-mon[32945]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:30:56.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:56 smithi031 ceph-mon[29167]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:30:56.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:56 smithi169 ceph-mon[33112]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:30:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:30:58 smithi031 ceph-mon[29167]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:30:58.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:30:58 smithi169 ceph-mon[33112]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:30:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:30:58 smithi040 ceph-mon[32945]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:31:00.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:00 smithi031 ceph-mon[29167]: pgmap v374: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T00:31:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:00 smithi169 ceph-mon[33112]: pgmap v374: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T00:31:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:00 smithi040 ceph-mon[32945]: pgmap v374: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T00:31:02.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:02 smithi031 ceph-mon[29167]: pgmap v375: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:02.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:02 smithi169 ceph-mon[33112]: pgmap v375: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:02 smithi040 ceph-mon[32945]: pgmap v375: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:04.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:04 smithi031 ceph-mon[29167]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:31:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:04 smithi169 ceph-mon[33112]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:31:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:04 smithi040 ceph-mon[32945]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:31:06.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:06 smithi031 ceph-mon[29167]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:06 smithi169 ceph-mon[33112]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:06 smithi040 ceph-mon[32945]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:08.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:08 smithi031 ceph-mon[29167]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:08.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:08 smithi169 ceph-mon[33112]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:08 smithi040 ceph-mon[32945]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T00:31:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.328+0000 7fe378563640 1 -- 172.21.15.31:0/2226818396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe370102250 msgr2=0x7fe370108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.328+0000 7fe378563640 1 --2- 172.21.15.31:0/2226818396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe370102250 0x7fe370108ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ca0 tx=0x7fe36802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- 172.21.15.31:0/2226818396 shutdown_connections 2024-04-25T00:31:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 --2- 172.21.15.31:0/2226818396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe370102250 0x7fe370108ae0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 --2- 172.21.15.31:0/2226818396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe370101890 0x7fe370101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 --2- 172.21.15.31:0/2226818396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3701006a0 0x7fe370100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- 172.21.15.31:0/2226818396 >> 172.21.15.31:0/2226818396 conn(0x7fe3700fbc10 msgr2=0x7fe3700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- 172.21.15.31:0/2226818396 shutdown_connections 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- 172.21.15.31:0/2226818396 wait complete. 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 Processor -- start 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- start start 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 0x7fe3701967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe370101890 0x7fe370196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370102250 0x7fe37019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe37010c1b0 con 0x7fe3701006a0 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe37010c030 con 0x7fe370102250 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe378563640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe37010c330 con 0x7fe370101890 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.329+0000 7fe3762ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 0x7fe3701967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 0x7fe3701967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56298/0 (socket says 172.21.15.31:56298) 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 -- 172.21.15.31:0/526836283 learned_addr learned my addr 172.21.15.31:0/526836283 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe376b00640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370102250 0x7fe37019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe375afe640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe370101890 0x7fe370196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 -- 172.21.15.31:0/526836283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe370101890 msgr2=0x7fe370196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe370101890 0x7fe370196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 -- 172.21.15.31:0/526836283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370102250 msgr2=0x7fe37019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370102250 0x7fe37019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 -- 172.21.15.31:0/526836283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37019e3e0 con 0x7fe3701006a0 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3762ff640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 0x7fe3701967c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00ead0 tx=0x7fe36c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3677fe640 1 -- 172.21.15.31:0/526836283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00bbd0 con 0x7fe3701006a0 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3677fe640 1 -- 172.21.15.31:0/526836283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c00bd70 con 0x7fe3701006a0 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe370068e50 con 0x7fe3701006a0 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe3677fe640 1 -- 172.21.15.31:0/526836283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c017410 con 0x7fe3701006a0 2024-04-25T00:31:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.330+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3700693f0 con 0x7fe3701006a0 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.333+0000 7fe3677fe640 1 -- 172.21.15.31:0/526836283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe36c01e020 con 0x7fe3701006a0 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.333+0000 7fe3677fe640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe35c0a2d50 0x7fe35c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.333+0000 7fe375afe640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe35c0a2d50 0x7fe35c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.333+0000 7fe3677fe640 1 -- 172.21.15.31:0/526836283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe36c0d0890 con 0x7fe3701006a0 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe375afe640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe35c0a2d50 0x7fe35c0a5210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe370197cd0 tx=0x7fe36000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 Processor -- start 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 -- start start 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24ede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ca2449cb0 con 0x564ca24eb9b0 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564ca23f25b0 con 0x564ca24e6840 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe37b278740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ca2445c70 con 0x564ca24e5e50 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.334+0000 7fe376b00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24ede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56302/0 (socket says 172.21.15.31:56302) 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 -- 172.21.15.31:0/944547713 learned_addr learned my addr 172.21.15.31:0/944547713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe375afe640 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe3762ff640 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 -- 172.21.15.31:0/944547713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 msgr2=0x564ca24e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 -- 172.21.15.31:0/944547713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 msgr2=0x564ca24eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 -- 172.21.15.31:0/944547713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca215e210 con 0x564ca24eb9b0 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe376b00640 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24ede40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ca0 tx=0x7fe368009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a8cc1af93f86e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe364bfe640 1 -- 172.21.15.31:0/944547713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368031710 con 0x564ca24eb9b0 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe364bfe640 1 -- 172.21.15.31:0/944547713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe368036030 con 0x564ca24eb9b0 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.335+0000 7fe364bfe640 1 -- 172.21.15.31:0/944547713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368004500 con 0x564ca24eb9b0 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 -- 172.21.15.31:0/944547713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 msgr2=0x564ca24ede40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24ede40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ca0 tx=0x7fe368009a20 comp rx=0 tx=0).stop 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 -- 172.21.15.31:0/944547713 shutdown_connections 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24ede40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 --2- 172.21.15.31:0/944547713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 -- 172.21.15.31:0/944547713 >> 172.21.15.31:0/944547713 conn(0x564ca24e14c0 msgr2=0x564ca24e18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 -- 172.21.15.31:0/944547713 shutdown_connections 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 -- 172.21.15.31:0/944547713 wait complete. 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.336+0000 7fe37b278740 1 -- start start 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 Processor -- start 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 0x564ca24f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e6840 0x564ca24f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ca24ee780 con 0x564ca24e5e50 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564ca2405b50 con 0x564ca24eb9b0 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ca24ee900 con 0x564ca24e6840 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 0x564ca24f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe37b278740 10 client.0 _ll_register_callbacks cb 0x564ca2440ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 0x564ca24f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56306/0 (socket says 172.21.15.31:56306) 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 -- 172.21.15.31:0/3122212640 learned_addr learned my addr 172.21.15.31:0/3122212640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe375afe640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e6840 0x564ca24f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe376b00640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 -- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e6840 msgr2=0x564ca24f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e6840 0x564ca24f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 -- 172.21.15.31:0/3122212640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 msgr2=0x564ca24f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 -- 172.21.15.31:0/3122212640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca222a620 con 0x564ca24e5e50 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe376b00640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe3762ff640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 0x564ca24f0060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe36c09cc60 tx=0x7fe36c0036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5421f4e3b24ff4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.337+0000 7fe359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe359ffb640 1 -- 172.21.15.31:0/3122212640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0a5940 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe359ffb640 1 -- 172.21.15.31:0/3122212640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe36c0bbba0 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe368036030 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe359ffb640 1 -- 172.21.15.31:0/3122212640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe36c0a1310 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 10 client.15423 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ca24faa20 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe359ffb640 1 -- 172.21.15.31:0/3122212640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe368036030 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe359ffb640 1 -- 172.21.15.31:0/3122212640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe36c098dd0 con 0x564ca24e5e50 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 20 client.15423 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca24fc160 0x564ca24fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 4 client.15423 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.338+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ca24571f0 con 0x564ca24fc160 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.339+0000 7fe375afe640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca24fc160 0x564ca24fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.339+0000 7fe375afe640 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca24fc160 0x564ca24fe5b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a44cbb56c53732 server_cookie=52e79c96fc15e449 in_seq=0 out_seq=0 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.339+0000 7fe359ffb640 10 client.15423 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe359ffb640 1 -- 172.21.15.31:0/3122212640 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ca24571f0 con 0x564ca24fc160 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe359ffb640 10 client.15423 handle_command_reply: tid=0 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 client.15423 shutdown 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 msgr2=0x564ca24f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 0x564ca24f0060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe36c09cc60 tx=0x7fe36c0036b0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 shutdown_connections 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca24fc160 0x564ca24fe5b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e6840 0x564ca24f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe359ffb640 0 client.15423 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 --2- 172.21.15.31:0/3122212640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e5e50 0x564ca24f0060 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 >> 172.21.15.31:0/3122212640 conn(0x564ca24e14c0 msgr2=0x564ca24e05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 shutdown_connections 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 1 -- 172.21.15.31:0/3122212640 wait complete. 2024-04-25T00:31:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.340+0000 7fe37b278740 20 client.15423 trim_cache size 0 max 16384 2024-04-25T00:31:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.369+0000 7fe37b278740 1 Processor -- start 2024-04-25T00:31:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 -- start start 2024-04-25T00:31:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a6950 con 0x564ca24eb9b0 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a8490 con 0x564ca24e6840 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe37b278740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a7020 con 0x564ca24e5e50 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe376b00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe376b00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56314/0 (socket says 172.21.15.31:56314) 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe376b00640 1 -- 172.21.15.31:0/3214668662 learned_addr learned my addr 172.21.15.31:0/3214668662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe375afe640 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 -- 172.21.15.31:0/3214668662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 msgr2=0x564ca24dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 -- 172.21.15.31:0/3214668662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 msgr2=0x564ca24e0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24e0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 -- 172.21.15.31:0/3214668662 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca24fec30 con 0x564ca24e5e50 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe375afe640 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe376b00640 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.370+0000 7fe3762ff640 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f0380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0058c0 tx=0x7fe36c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f3e53512cbbd08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe35bfff640 1 -- 172.21.15.31:0/3214668662 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c09caa0 con 0x564ca24e5e50 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe35bfff640 1 -- 172.21.15.31:0/3214668662 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c0a5bd0 con 0x564ca24e5e50 2024-04-25T00:31:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe35bfff640 1 -- 172.21.15.31:0/3214668662 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c098cb0 con 0x564ca24e5e50 2024-04-25T00:31:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 -- 172.21.15.31:0/3214668662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 msgr2=0x564ca24f0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f0380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0058c0 tx=0x7fe36c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 -- 172.21.15.31:0/3214668662 shutdown_connections 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24eb9b0 0x564ca24e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24e6840 0x564ca24dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 --2- 172.21.15.31:0/3214668662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f0380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 -- 172.21.15.31:0/3214668662 >> 172.21.15.31:0/3214668662 conn(0x564ca24e14c0 msgr2=0x564ca24efc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.371+0000 7fe37b278740 1 -- 172.21.15.31:0/3214668662 shutdown_connections 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 -- 172.21.15.31:0/3214668662 wait complete. 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 -- start start 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 Processor -- start 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 0x564ca24f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a7dc0 con 0x564ca24e6840 2024-04-25T00:31:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a76f0 con 0x564ca24eb9b0 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a4930 con 0x564ca24e5e50 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe37b278740 10 client.0 _ll_register_callbacks cb 0x7fe36c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 0x564ca24f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 0x564ca24f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56324/0 (socket says 172.21.15.31:56324) 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe3762ff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 -- 172.21.15.31:0/1694639107 learned_addr learned my addr 172.21.15.31:0/1694639107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe376b00640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 -- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 msgr2=0x564ca24f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 -- 172.21.15.31:0/1694639107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 msgr2=0x564ca24f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.372+0000 7fe375afe640 1 -- 172.21.15.31:0/1694639107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca24f5b40 con 0x564ca24e6840 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe376b00640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe375afe640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 0x564ca24f4540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b370 tx=0x7fe3600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a41e3ab2e9a75a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe359ffb640 1 -- 172.21.15.31:0/1694639107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36002b520 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe359ffb640 1 -- 172.21.15.31:0/1694639107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe36002b6c0 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36c0a5bd0 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe359ffb640 1 -- 172.21.15.31:0/1694639107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe36003b050 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe37b278740 10 client.15429 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ca2566b40 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe359ffb640 1 -- 172.21.15.31:0/1694639107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c0a5bd0 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.373+0000 7fe359ffb640 1 -- 172.21.15.31:0/1694639107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe36002a410 con 0x564ca24e6840 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe37b278740 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe37b278740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe37b278740 20 client.15429 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe37b278740 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca25673b0 0x564ca25697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe37b278740 4 client.15429 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ca2451540 con 0x564ca25673b0 2024-04-25T00:31:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe3762ff640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca25673b0 0x564ca25697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe359ffb640 10 client.15429 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.374+0000 7fe3762ff640 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca25673b0 0x564ca25697d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd67c203d848f0a1 server_cookie=338e3638acdb4e03 in_seq=0 out_seq=0 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe359ffb640 1 -- 172.21.15.31:0/1694639107 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ca2451540 con 0x564ca25673b0 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe359ffb640 10 client.15429 handle_command_reply: tid=0 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 client.15429 shutdown 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 msgr2=0x564ca24f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 0x564ca24f4540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b370 tx=0x7fe3600046c0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 shutdown_connections 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564ca25673b0 0x564ca25697d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ca24eb9b0 0x564ca24f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe359ffb640 0 client.15429 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ca24e6840 0x564ca24f4540 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 --2- 172.21.15.31:0/1694639107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ca24e5e50 0x564ca24f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 >> 172.21.15.31:0/1694639107 conn(0x564ca24e14c0 msgr2=0x564ca24e05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 shutdown_connections 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 1 -- 172.21.15.31:0/1694639107 wait complete. 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.378+0000 7fe37b278740 20 client.15429 trim_cache size 0 max 16384 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:31:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe35c0a2d50 msgr2=0x7fe35c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe35c0a2d50 0x7fe35c0a5210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe370197cd0 tx=0x7fe36000b590 comp rx=0 tx=0).stop 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 msgr2=0x7fe3701967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 0x7fe3701967c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00ead0 tx=0x7fe36c00ef90 comp rx=0 tx=0).stop 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 shutdown_connections 2024-04-25T00:31:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370102250 0x7fe37019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe370101890 0x7fe370196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe35c0a2d50 0x7fe35c0a5210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 --2- 172.21.15.31:0/526836283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3701006a0 0x7fe3701967c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 >> 172.21.15.31:0/526836283 conn(0x7fe3700fbc10 msgr2=0x7fe370104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 shutdown_connections 2024-04-25T00:31:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:09.379+0000 7fe378563640 1 -- 172.21.15.31:0/526836283 wait complete. 2024-04-25T00:31:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:10 smithi031 ceph-mon[29167]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:31:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:10 smithi169 ceph-mon[33112]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:31:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:10 smithi040 ceph-mon[32945]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:31:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:12 smithi031 ceph-mon[29167]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:31:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:12 smithi169 ceph-mon[33112]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:31:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:12 smithi040 ceph-mon[32945]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:31:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:14 smithi031 ceph-mon[29167]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:14.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:14 smithi169 ceph-mon[33112]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:14 smithi040 ceph-mon[32945]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:16.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:16 smithi031 ceph-mon[29167]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:16 smithi169 ceph-mon[33112]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:16.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:16 smithi040 ceph-mon[32945]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:18 smithi031 ceph-mon[29167]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:18 smithi169 ceph-mon[33112]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:18 smithi040 ceph-mon[32945]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:20 smithi031 ceph-mon[29167]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:20 smithi169 ceph-mon[33112]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:20 smithi040 ceph-mon[32945]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 -- 172.21.15.31:0/1689965259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570102250 msgr2=0x7fa570108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 --2- 172.21.15.31:0/1689965259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570102250 0x7fa570108ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa568009ca0 tx=0x7fa56802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 -- 172.21.15.31:0/1689965259 shutdown_connections 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 --2- 172.21.15.31:0/1689965259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570102250 0x7fa570108ae0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 --2- 172.21.15.31:0/1689965259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa570101890 0x7fa570101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 --2- 172.21.15.31:0/1689965259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5701006a0 0x7fa570100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 -- 172.21.15.31:0/1689965259 >> 172.21.15.31:0/1689965259 conn(0x7fa5700fbc10 msgr2=0x7fa5700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 -- 172.21.15.31:0/1689965259 shutdown_connections 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.482+0000 7fa575edf640 1 -- 172.21.15.31:0/1689965259 wait complete. 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 Processor -- start 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 -- start start 2024-04-25T00:31:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5701006a0 0x7fa570196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 0x7fa570196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa570102250 0x7fa57019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010c200 con 0x7fa570101890 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010c080 con 0x7fa570102250 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa575edf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010c380 con 0x7fa5701006a0 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa56effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 0x7fa570196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.483+0000 7fa56effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 0x7fa570196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55326/0 (socket says 172.21.15.31:55326) 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 -- 172.21.15.31:0/3202919994 learned_addr learned my addr 172.21.15.31:0/3202919994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5701006a0 0x7fa570196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56ffff640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa570102250 0x7fa57019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 -- 172.21.15.31:0/3202919994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5701006a0 msgr2=0x7fa570196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5701006a0 0x7fa570196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 -- 172.21.15.31:0/3202919994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa570102250 msgr2=0x7fa57019dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa570102250 0x7fa57019dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 -- 172.21.15.31:0/3202919994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57019e370 con 0x7fa570101890 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56effd640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 0x7fa570196c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c890 tx=0x7fa56400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56cff9640 1 -- 172.21.15.31:0/3202919994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564004220 con 0x7fa570101890 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57019e660 con 0x7fa570101890 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56cff9640 1 -- 172.21.15.31:0/3202919994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5640043c0 con 0x7fa570101890 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa56cff9640 1 -- 172.21.15.31:0/3202919994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564004a00 con 0x7fa570101890 2024-04-25T00:31:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.484+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57019ec00 con 0x7fa570101890 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa56cff9640 1 -- 172.21.15.31:0/3202919994 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa564004ba0 con 0x7fa570101890 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 Processor -- start 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 -- start start 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7ba060 0x55a99b7ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 0x55a99b7c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a99b71bcb0 con 0x55a99b7bfbc0 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a99b6c45b0 con 0x55a99b7ba060 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.487+0000 7fa578bdf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a99b717c70 con 0x55a99b7baa50 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 0x55a99b7c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56cff9640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa5580a2cd0 0x7fa5580a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 0x55a99b7c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55340/0 (socket says 172.21.15.31:55340) 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 -- 172.21.15.31:0/2829578345 learned_addr learned my addr 172.21.15.31:0/2829578345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56ffff640 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7ba060 0x55a99b7ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56ffff640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa5580a2cd0 0x7fa5580a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56cff9640 1 -- 172.21.15.31:0/3202919994 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa5640ceef0 con 0x7fa570101890 2024-04-25T00:31:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 -- 172.21.15.31:0/2829578345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 msgr2=0x55a99b7bf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7bf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 -- 172.21.15.31:0/2829578345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7ba060 msgr2=0x55a99b7ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7ba060 0x55a99b7ba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56effd640 1 -- 172.21.15.31:0/2829578345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99b430210 con 0x55a99b7bfbc0 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.488+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.489+0000 7fa56effd640 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 0x55a99b7c2050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa564004610 tx=0x7fa56400a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30eef9466ed0349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.489+0000 7fa56ffff640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa5580a2cd0 0x7fa5580a5190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5680318d0 tx=0x7fa56802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.489+0000 7fa555ffb640 1 -- 172.21.15.31:0/2829578345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56401d070 con 0x55a99b7bfbc0 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.489+0000 7fa555ffb640 1 -- 172.21.15.31:0/2829578345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa56409b5c0 con 0x55a99b7bfbc0 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.489+0000 7fa555ffb640 1 -- 172.21.15.31:0/2829578345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564097c30 con 0x55a99b7bfbc0 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 -- 172.21.15.31:0/2829578345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 msgr2=0x55a99b7c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 0x55a99b7c2050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa564004610 tx=0x7fa56400a5e0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 -- 172.21.15.31:0/2829578345 shutdown_connections 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7bfbc0 0x55a99b7c2050 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2829578345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7ba060 0x55a99b7ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 -- 172.21.15.31:0/2829578345 >> 172.21.15.31:0/2829578345 conn(0x55a99b7b34c0 msgr2=0x55a99b7b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 -- 172.21.15.31:0/2829578345 shutdown_connections 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 -- 172.21.15.31:0/2829578345 wait complete. 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 -- start start 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.490+0000 7fa578bdf740 1 Processor -- start 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a99b7b7d50 con 0x55a99b7ba060 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a99b6d7b50 con 0x55a99b7cbe50 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a99b7b7ed0 con 0x55a99b7baa50 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa578bdf740 10 client.0 _ll_register_callbacks cb 0x55a99b712ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55344/0 (socket says 172.21.15.31:55344) 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 -- 172.21.15.31:0/3780809539 learned_addr learned my addr 172.21.15.31:0/3780809539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56ffff640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 -- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 msgr2=0x55a99b7c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 -- 172.21.15.31:0/3780809539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 msgr2=0x55a99b7ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56f7fe640 1 -- 172.21.15.31:0/3780809539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99b4fc620 con 0x55a99b7ba060 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56effd640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.491+0000 7fa56ffff640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c6ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa560010830 tx=0x7fa560010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd67aed06e709cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa54effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa54effd640 1 -- 172.21.15.31:0/3780809539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560004260 con 0x55a99b7ba060 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa54effd640 1 -- 172.21.15.31:0/3780809539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa560004400 con 0x55a99b7ba060 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56409b5c0 con 0x55a99b7ba060 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa54effd640 1 -- 172.21.15.31:0/3780809539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa56000f980 con 0x55a99b7ba060 2024-04-25T00:31:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa578bdf740 10 client.15441 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a99b7d16a0 con 0x55a99b7ba060 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa54effd640 1 -- 172.21.15.31:0/3780809539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa56409b5c0 con 0x55a99b7ba060 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa54effd640 1 -- 172.21.15.31:0/3780809539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa560024430 con 0x55a99b7ba060 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.492+0000 7fa578bdf740 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa578bdf740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa578bdf740 20 client.15441 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa578bdf740 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b7d2d20 0x55a99b7d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa578bdf740 4 client.15441 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a99b7291f0 con 0x55a99b7d2d20 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa56effd640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b7d2d20 0x55a99b7d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa56effd640 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b7d2d20 0x55a99b7d5150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e7ae74b93e3d56c server_cookie=5973b6d9920589b9 in_seq=0 out_seq=0 2024-04-25T00:31:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.493+0000 7fa54effd640 10 client.15441 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.494+0000 7fa54effd640 1 -- 172.21.15.31:0/3780809539 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a99b7291f0 con 0x55a99b7d2d20 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.494+0000 7fa54effd640 10 client.15441 handle_command_reply: tid=0 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.494+0000 7fa578bdf740 1 client.15441 shutdown 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.494+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 msgr2=0x55a99b7c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.494+0000 7fa578bdf740 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c6ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa560010830 tx=0x7fa560010cf0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 shutdown_connections 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b7d2d20 0x55a99b7d5150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa54effd640 0 client.15441 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 --2- 172.21.15.31:0/3780809539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c6ef0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 >> 172.21.15.31:0/3780809539 conn(0x55a99b7b34c0 msgr2=0x55a99b7b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 shutdown_connections 2024-04-25T00:31:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 1 -- 172.21.15.31:0/3780809539 wait complete. 2024-04-25T00:31:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.495+0000 7fa578bdf740 20 client.15441 trim_cache size 0 max 16384 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 Processor -- start 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 -- start start 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7dafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a69f0 con 0x55a99b7ba060 2024-04-25T00:31:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a8530 con 0x55a99b7cbe50 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa578bdf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a70c0 con 0x55a99b7baa50 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.513+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55350/0 (socket says 172.21.15.31:55350) 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 -- 172.21.15.31:0/607949406 learned_addr learned my addr 172.21.15.31:0/607949406 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56effd640 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56ffff640 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7dafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 -- 172.21.15.31:0/607949406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 msgr2=0x55a99b7daa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7daa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 -- 172.21.15.31:0/607949406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 msgr2=0x55a99b7dafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7dafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 -- 172.21.15.31:0/607949406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99b7cf6e0 con 0x55a99b7ba060 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56effd640 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c7390 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5600248d0 tx=0x7fa560025f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b71ae541b11ffbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa5557fa640 1 -- 172.21.15.31:0/607949406 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560024430 con 0x55a99b7ba060 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa5557fa640 1 -- 172.21.15.31:0/607949406 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5600245d0 con 0x55a99b7ba060 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.514+0000 7fa5557fa640 1 -- 172.21.15.31:0/607949406 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560003ab0 con 0x55a99b7ba060 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 -- 172.21.15.31:0/607949406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 msgr2=0x55a99b7c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c7390 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5600248d0 tx=0x7fa560025f20 comp rx=0 tx=0).stop 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 -- 172.21.15.31:0/607949406 shutdown_connections 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7cbe50 0x55a99b7dafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7baa50 0x55a99b7daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 --2- 172.21.15.31:0/607949406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7ba060 0x55a99b7c7390 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 -- 172.21.15.31:0/607949406 >> 172.21.15.31:0/607949406 conn(0x55a99b7b34c0 msgr2=0x55a99b7c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 -- 172.21.15.31:0/607949406 shutdown_connections 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 -- 172.21.15.31:0/607949406 wait complete. 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.515+0000 7fa578bdf740 1 -- start start 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 Processor -- start 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7ba060 0x55a99b7e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7baa50 0x55a99b7e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 0x55a99b7e5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a7e60 con 0x55a99b7cbe50 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a7790 con 0x55a99b7baa50 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a49d0 con 0x55a99b7ba060 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa578bdf740 10 client.0 _ll_register_callbacks cb 0x7fa56000a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 0x55a99b7e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 0x55a99b7e5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55358/0 (socket says 172.21.15.31:55358) 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa56ffff640 1 -- 172.21.15.31:0/2122190629 learned_addr learned my addr 172.21.15.31:0/2122190629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7ba060 0x55a99b7e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.516+0000 7fa56effd640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7baa50 0x55a99b7e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56ffff640 1 -- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7ba060 msgr2=0x55a99b7e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56ffff640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7ba060 0x55a99b7e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56ffff640 1 -- 172.21.15.31:0/2122190629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7baa50 msgr2=0x55a99b7e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56ffff640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7baa50 0x55a99b7e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56ffff640 1 -- 172.21.15.31:0/2122190629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99b7e5a00 con 0x55a99b7cbe50 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56effd640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7baa50 0x55a99b7e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa56ffff640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 0x55a99b7e5410 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa568009ca0 tx=0x7fa5680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0ea5a3910158d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa54effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa54effd640 1 -- 172.21.15.31:0/2122190629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568030990 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa54effd640 1 -- 172.21.15.31:0/2122190629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa568030b30 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5600245d0 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa54effd640 1 -- 172.21.15.31:0/2122190629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa56805e050 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa578bdf740 10 client.15447 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a99b839030 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.517+0000 7fa54effd640 1 -- 172.21.15.31:0/2122190629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5600245d0 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa54effd640 1 -- 172.21.15.31:0/2122190629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa56803b410 con 0x55a99b7cbe50 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa578bdf740 10 client.15447 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa578bdf740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa578bdf740 20 client.15447 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b839a30 0x55a99b83be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa578bdf740 4 client.15447 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a99b723540 con 0x55a99b839a30 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.518+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b839a30 0x55a99b83be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.519+0000 7fa56f7fe640 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b839a30 0x55a99b83be50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9671f91fd1b8ba server_cookie=1c711f6e1bc90045 in_seq=0 out_seq=0 2024-04-25T00:31:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.519+0000 7fa54effd640 10 client.15447 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.522+0000 7fa54effd640 1 -- 172.21.15.31:0/2122190629 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a99b723540 con 0x55a99b839a30 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa54effd640 10 client.15447 handle_command_reply: tid=0 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 client.15447 shutdown 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 msgr2=0x55a99b7e5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 0x55a99b7e5410 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa568009ca0 tx=0x7fa5680095c0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 shutdown_connections 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a99b839a30 0x55a99b83be50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a99b7cbe50 0x55a99b7e5410 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a99b7baa50 0x55a99b7e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa54effd640 0 client.15447 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 --2- 172.21.15.31:0/2122190629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a99b7ba060 0x55a99b7e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 >> 172.21.15.31:0/2122190629 conn(0x55a99b7b34c0 msgr2=0x55a99b7b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 shutdown_connections 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 1 -- 172.21.15.31:0/2122190629 wait complete. 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.523+0000 7fa578bdf740 20 client.15447 trim_cache size 0 max 16384 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:31:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:31:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa5580a2cd0 msgr2=0x7fa5580a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa5580a2cd0 0x7fa5580a5190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5680318d0 tx=0x7fa56802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 msgr2=0x7fa570196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 0x7fa570196c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c890 tx=0x7fa56400cd50 comp rx=0 tx=0).stop 2024-04-25T00:31:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 shutdown_connections 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa570102250 0x7fa57019dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa5580a2cd0 0x7fa5580a5190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa570101890 0x7fa570196c90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 --2- 172.21.15.31:0/3202919994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5701006a0 0x7fa570196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 >> 172.21.15.31:0/3202919994 conn(0x7fa5700fbc10 msgr2=0x7fa570104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 shutdown_connections 2024-04-25T00:31:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:22.524+0000 7fa575edf640 1 -- 172.21.15.31:0/3202919994 wait complete. 2024-04-25T00:31:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:22 smithi031 ceph-mon[29167]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:22.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:22 smithi169 ceph-mon[33112]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:22 smithi040 ceph-mon[32945]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:24 smithi031 ceph-mon[29167]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T00:31:24.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:24 smithi169 ceph-mon[33112]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T00:31:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:24 smithi040 ceph-mon[32945]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T00:31:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:26 smithi031 ceph-mon[29167]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:26 smithi169 ceph-mon[33112]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:26 smithi040 ceph-mon[32945]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:28.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:28 smithi031 ceph-mon[29167]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:28 smithi169 ceph-mon[33112]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:28 smithi040 ceph-mon[32945]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:30 smithi031 ceph-mon[29167]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:30 smithi169 ceph-mon[33112]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:30 smithi040 ceph-mon[32945]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:32 smithi031 ceph-mon[29167]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:32.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:32 smithi169 ceph-mon[33112]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:32 smithi040 ceph-mon[32945]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:34 smithi031 ceph-mon[29167]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:34 smithi169 ceph-mon[33112]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:34 smithi040 ceph-mon[32945]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 -- 172.21.15.31:0/277161358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 msgr2=0x7f3c08100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 --2- 172.21.15.31:0/277161358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08100250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009900 tx=0x7f3bf802a410 comp rx=0 tx=0).stop 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 -- 172.21.15.31:0/277161358 shutdown_connections 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 --2- 172.21.15.31:0/277161358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c08101130 0x7f3c081015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 --2- 172.21.15.31:0/277161358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c08100790 0x7f3c08100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 --2- 172.21.15.31:0/277161358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08100250 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 -- 172.21.15.31:0/277161358 >> 172.21.15.31:0/277161358 conn(0x7f3c080fba30 msgr2=0x7f3c080fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 -- 172.21.15.31:0/277161358 shutdown_connections 2024-04-25T00:31:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.651+0000 7f3c0759e640 1 -- 172.21.15.31:0/277161358 wait complete. 2024-04-25T00:31:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 Processor -- start 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 -- start start 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c08100790 0x7f3c08199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c08101130 0x7f3c081a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810f2d0 con 0x7f3c080ffe30 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810f150 con 0x7f3c08101130 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810f450 con 0x7f3c08100790 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37864/0 (socket says 172.21.15.31:37864) 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c0659c640 1 -- 172.21.15.31:0/3651564685 learned_addr learned my addr 172.21.15.31:0/3651564685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.652+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c08100790 0x7f3c08199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c08101130 0x7f3c081a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0659c640 1 -- 172.21.15.31:0/3651564685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c08100790 msgr2=0x7f3c08199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0659c640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c08100790 0x7f3c08199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0659c640 1 -- 172.21.15.31:0/3651564685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c08101130 msgr2=0x7f3c081a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0659c640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c08101130 0x7f3c081a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0659c640 1 -- 172.21.15.31:0/3651564685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c081a1650 con 0x7f3c080ffe30 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0659c640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08199a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8004770 tx=0x7f3bf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3bf77fe640 1 -- 172.21.15.31:0/3651564685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf803f030 con 0x7f3c080ffe30 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c081a18e0 con 0x7f3c080ffe30 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3bf77fe640 1 -- 172.21.15.31:0/3651564685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bf8003cb0 con 0x7f3c080ffe30 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3bf77fe640 1 -- 172.21.15.31:0/3651564685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8035070 con 0x7f3c080ffe30 2024-04-25T00:31:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.653+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c081a1dd0 con 0x7f3c080ffe30 2024-04-25T00:31:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 Processor -- start 2024-04-25T00:31:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 -- start start 2024-04-25T00:31:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x562814651450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814656520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814656b90 0x562814659020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5628145b4cb0 con 0x562814651030 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56281455d5b0 con 0x562814656b90 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5628145b0c70 con 0x562814651a20 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c06d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x562814651450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.656+0000 7f3c06d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x562814651450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37878/0 (socket says 172.21.15.31:37878) 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c06d9d640 1 -- 172.21.15.31:0/3440842638 learned_addr learned my addr 172.21.15.31:0/3440842638 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814656b90 0x562814659020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3bf77fe640 1 -- 172.21.15.31:0/3651564685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3bf8042030 con 0x7f3c080ffe30 2024-04-25T00:31:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 -- 172.21.15.31:0/3440842638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 msgr2=0x562814656520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814656520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 -- 172.21.15.31:0/3440842638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 msgr2=0x562814651450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x562814651450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 -- 172.21.15.31:0/3440842638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628142c9210 con 0x562814656b90 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814656b90 0x562814659020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005fd0 tx=0x7f3bfc00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c83cb870f4145b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3bf4ff9640 1 -- 172.21.15.31:0/3440842638 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003bd0 con 0x562814656b90 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.657+0000 7f3bf4ff9640 1 -- 172.21.15.31:0/3440842638 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc015070 con 0x562814656b90 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3bf77fe640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3bf00a2c10 0x7f3bf00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3bf77fe640 1 -- 172.21.15.31:0/3651564685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3bf80efb90 con 0x7f3c080ffe30 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3bf4ff9640 1 -- 172.21.15.31:0/3440842638 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004d60 con 0x562814656b90 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 -- 172.21.15.31:0/3440842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814656b90 msgr2=0x562814659020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814656b90 0x562814659020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005fd0 tx=0x7f3bfc00bb20 comp rx=0 tx=0).stop 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3bf00a2c10 0x7f3bf00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 -- 172.21.15.31:0/3440842638 shutdown_connections 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814656b90 0x562814659020 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814656520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/3440842638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x562814651450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.658+0000 7f3c0fbef740 1 -- 172.21.15.31:0/3440842638 >> 172.21.15.31:0/3440842638 conn(0x56281464c6a0 msgr2=0x56281464caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.659+0000 7f3c0fbef740 1 -- 172.21.15.31:0/3440842638 shutdown_connections 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.659+0000 7f3c0fbef740 1 -- 172.21.15.31:0/3440842638 wait complete. 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.659+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3bf00a2c10 0x7f3bf00a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x562814652280 tx=0x7f3c00009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.659+0000 7f3c0fbef740 1 -- start start 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.659+0000 7f3c0fbef740 1 Processor -- start 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.659+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651030 0x56281465b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651a20 0x56281465b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 0x562814662930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562814659b90 con 0x562814656b90 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562814570b50 con 0x562814651a20 2024-04-25T00:31:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562814659d10 con 0x562814651030 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0fbef740 10 client.0 _ll_register_callbacks cb 0x5628145abca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 0x562814662930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 0x562814662930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37884/0 (socket says 172.21.15.31:37884) 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 -- 172.21.15.31:0/165103522 learned_addr learned my addr 172.21.15.31:0/165103522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c0659c640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651030 0x56281465b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651a20 0x56281465b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 -- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651030 msgr2=0x56281465b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651030 0x56281465b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 -- 172.21.15.31:0/165103522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651a20 msgr2=0x56281465b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651a20 0x56281465b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 -- 172.21.15.31:0/165103522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562814395620 con 0x562814656b90 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 0x562814662930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3c000128b0 tx=0x7f3c00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04e0ee0482f9e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.660+0000 7f3bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3bedffb640 1 -- 172.21.15.31:0/165103522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00004d20 con 0x562814656b90 2024-04-25T00:31:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc015070 con 0x562814656b90 2024-04-25T00:31:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3bedffb640 1 -- 172.21.15.31:0/165103522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3c0000eae0 con 0x562814656b90 2024-04-25T00:31:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3bedffb640 1 -- 172.21.15.31:0/165103522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c0000ece0 con 0x562814656b90 2024-04-25T00:31:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3c0fbef740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562814665dc0 con 0x562814656b90 2024-04-25T00:31:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.661+0000 7f3bedffb640 1 -- 172.21.15.31:0/165103522 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc015070 con 0x562814656b90 2024-04-25T00:31:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3bedffb640 1 -- 172.21.15.31:0/165103522 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c00030070 con 0x562814656b90 2024-04-25T00:31:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0fbef740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0fbef740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0fbef740 20 client.15459 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814667500 0x562814669940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0fbef740 4 client.15459 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628145c21f0 con 0x562814667500 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.662+0000 7f3c0659c640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814667500 0x562814669940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.663+0000 7f3c0659c640 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814667500 0x562814669940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4dfaf31ff7f45 server_cookie=8ad8e0c91dba2f11 in_seq=0 out_seq=0 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.663+0000 7f3bedffb640 10 client.15459 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.663+0000 7f3bedffb640 1 -- 172.21.15.31:0/165103522 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628145c21f0 con 0x562814667500 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.663+0000 7f3bedffb640 10 client.15459 handle_command_reply: tid=0 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.663+0000 7f3c0fbef740 1 client.15459 shutdown 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 msgr2=0x562814662930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 0x562814662930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3c000128b0 tx=0x7f3c00004a40 comp rx=0 tx=0).stop 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 shutdown_connections 2024-04-25T00:31:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814667500 0x562814669940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3bedffb640 0 client.15459 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814656b90 0x562814662930 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651a20 0x56281465b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/165103522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651030 0x56281465b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 >> 172.21.15.31:0/165103522 conn(0x56281464c6a0 msgr2=0x56281464b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 shutdown_connections 2024-04-25T00:31:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 1 -- 172.21.15.31:0/165103522 wait complete. 2024-04-25T00:31:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.664+0000 7f3c0fbef740 20 client.15459 trim_cache size 0 max 16384 2024-04-25T00:31:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.679+0000 7f3c0fbef740 1 Processor -- start 2024-04-25T00:31:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.679+0000 7f3c0fbef740 1 -- start start 2024-04-25T00:31:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56281464c6a0 0x56281464b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x5628146628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x56281464b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081aacd0 con 0x562814651030 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ac810 con 0x56281464c6a0 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ab3a0 con 0x562814651a20 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x5628146628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x5628146628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37886/0 (socket says 172.21.15.31:37886) 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 -- 172.21.15.31:0/388134994 learned_addr learned my addr 172.21.15.31:0/388134994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0659c640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56281464c6a0 0x56281464b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x56281464b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 -- 172.21.15.31:0/388134994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 msgr2=0x56281464b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x56281464b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 -- 172.21.15.31:0/388134994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56281464c6a0 msgr2=0x56281464b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56281464c6a0 0x56281464b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 -- 172.21.15.31:0/388134994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562814663db0 con 0x562814651030 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x56281464b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c0659c640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56281464c6a0 0x56281464b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.680+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x5628146628f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc004580 tx=0x7f3bfc004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4e500fa19d6e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3beffff640 1 -- 172.21.15.31:0/388134994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc028070 con 0x562814651030 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3beffff640 1 -- 172.21.15.31:0/388134994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc015040 con 0x562814651030 2024-04-25T00:31:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3beffff640 1 -- 172.21.15.31:0/388134994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc019420 con 0x562814651030 2024-04-25T00:31:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 -- 172.21.15.31:0/388134994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 msgr2=0x5628146628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x5628146628f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc004580 tx=0x7f3bfc004140 comp rx=0 tx=0).stop 2024-04-25T00:31:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 -- 172.21.15.31:0/388134994 shutdown_connections 2024-04-25T00:31:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x56281464b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562814651030 0x5628146628f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/388134994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56281464c6a0 0x56281464b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 -- 172.21.15.31:0/388134994 >> 172.21.15.31:0/388134994 conn(0x562814656660 msgr2=0x562814652ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 -- 172.21.15.31:0/388134994 shutdown_connections 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.681+0000 7f3c0fbef740 1 -- 172.21.15.31:0/388134994 wait complete. 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 -- start start 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 Processor -- start 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 0x56281465f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651030 0x56281465ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814660f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ac140 con 0x56281464c6a0 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081aba70 con 0x562814651030 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081a8cb0 con 0x562814651a20 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0fbef740 10 client.0 _ll_register_callbacks cb 0x7f3c00009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 0x56281465f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 0x56281465f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37894/0 (socket says 172.21.15.31:37894) 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0659c640 1 -- 172.21.15.31:0/80804566 learned_addr learned my addr 172.21.15.31:0/80804566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651030 0x56281465ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c06d9d640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814660f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0659c640 1 -- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 msgr2=0x562814660f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.682+0000 7f3c0659c640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814660f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0659c640 1 -- 172.21.15.31:0/80804566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651030 msgr2=0x56281465ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0659c640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651030 0x56281465ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0659c640 1 -- 172.21.15.31:0/80804566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562814661530 con 0x56281464c6a0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651030 0x56281465ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0659c640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 0x56281465f9f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80b8c30 tx=0x7f3bf802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569e3064fc7f0ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3bedffb640 1 -- 172.21.15.31:0/80804566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8049040 con 0x56281464c6a0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3bedffb640 1 -- 172.21.15.31:0/80804566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3bf80ed760 con 0x56281464c6a0 2024-04-25T00:31:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc015040 con 0x56281464c6a0 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3bedffb640 1 -- 172.21.15.31:0/80804566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bf80c0610 con 0x56281464c6a0 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0fbef740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562814662820 con 0x56281464c6a0 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.683+0000 7f3bedffb640 1 -- 172.21.15.31:0/80804566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc015040 con 0x56281464c6a0 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3bedffb640 1 -- 172.21.15.31:0/80804566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf803b460 con 0x56281464c6a0 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c0fbef740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c0fbef740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c0fbef740 20 client.15465 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814656b90 0x5628146cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c0fbef740 4 client.15465 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628145bc540 con 0x562814656b90 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814656b90 0x5628146cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3bedffb640 10 client.15465 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.684+0000 7f3c05d9b640 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814656b90 0x5628146cfe70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c7b596e9a71897 server_cookie=b3f9597d9ee1dc93 in_seq=0 out_seq=0 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3bedffb640 1 -- 172.21.15.31:0/80804566 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628145bc540 con 0x562814656b90 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3bedffb640 10 client.15465 handle_command_reply: tid=0 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 client.15465 shutdown 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 msgr2=0x56281465f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 0x56281465f9f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80b8c30 tx=0x7f3bf802a410 comp rx=0 tx=0).stop 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 shutdown_connections 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x562814656b90 0x5628146cfe70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562814651a20 0x562814660f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562814651030 0x56281465ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3bedffb640 0 client.15465 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 --2- 172.21.15.31:0/80804566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56281464c6a0 0x56281465f9f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 >> 172.21.15.31:0/80804566 conn(0x562814656660 msgr2=0x56281464f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 shutdown_connections 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 1 -- 172.21.15.31:0/80804566 wait complete. 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.688+0000 7f3c0fbef740 20 client.15465 trim_cache size 0 max 16384 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:31:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:31:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3bf00a2c10 msgr2=0x7f3bf00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3bf00a2c10 0x7f3bf00a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x562814652280 tx=0x7f3c00009760 comp rx=0 tx=0).stop 2024-04-25T00:31:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 msgr2=0x7f3c08199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08199a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8004770 tx=0x7f3bf80042f0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 shutdown_connections 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c08101130 0x7f3c081a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c08100790 0x7f3c08199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3bf00a2c10 0x7f3bf00a50d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 --2- 172.21.15.31:0/3651564685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c080ffe30 0x7f3c08199a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 >> 172.21.15.31:0/3651564685 conn(0x7f3c080fba30 msgr2=0x7f3c080fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 shutdown_connections 2024-04-25T00:31:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:35.689+0000 7f3c0759e640 1 -- 172.21.15.31:0/3651564685 wait complete. 2024-04-25T00:31:36.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:36 smithi031 ceph-mon[29167]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:31:36.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:36 smithi169 ceph-mon[33112]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:31:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:36 smithi040 ceph-mon[32945]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:31:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:38 smithi031 ceph-mon[29167]: pgmap v393: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:31:38.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:38 smithi169 ceph-mon[33112]: pgmap v393: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:31:38.938 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:38 smithi040 ceph-mon[32945]: pgmap v393: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:31:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:40 smithi031 ceph-mon[29167]: pgmap v394: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:40.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:40 smithi169 ceph-mon[33112]: pgmap v394: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:40 smithi040 ceph-mon[32945]: pgmap v394: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:42.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:42 smithi031 ceph-mon[29167]: pgmap v395: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:42 smithi169 ceph-mon[33112]: pgmap v395: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:42.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:42 smithi040 ceph-mon[32945]: pgmap v395: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T00:31:44.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:44 smithi031 ceph-mon[29167]: pgmap v396: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:44.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:44 smithi031 ceph-mon[29167]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T00:31:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:44 smithi169 ceph-mon[33112]: pgmap v396: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:44 smithi169 ceph-mon[33112]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T00:31:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:44 smithi040 ceph-mon[32945]: pgmap v396: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:31:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:44 smithi040 ceph-mon[32945]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T00:31:46.451 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 97% (12155/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T00:31:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:46 smithi031 ceph-mon[29167]: pgmap v397: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:46 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:46 smithi169 ceph-mon[33112]: pgmap v397: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:46 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:46 smithi040 ceph-mon[32945]: pgmap v397: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:46 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:47.039 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T00:31:47.039 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.040 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T00:31:47.041 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T00:31:47.041 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T00:31:47.043 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T00:31:47.109 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T00:31:47.112 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:31:47.112 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T00:31:47.802 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T00:31:47.854 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T00:31:47.895 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:31:47.896 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:31:47.896 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T00:31:47.936 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:31:47.939 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:31:47.939 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T00:31:47.995 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-25T00:31:47.997 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-25T00:31:47.997 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T00:31:48.061 INFO:tasks.workunit.client.0.smithi031.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T00:31:48.061 INFO:tasks.workunit.client.0.smithi031.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-25T00:31:48.062 INFO:tasks.workunit.client.0.smithi031.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-25T00:31:48.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:48.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:47 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:48.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:47 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:48.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:48.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:47 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 -- 172.21.15.31:0/436084720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e48107530 msgr2=0x7f1e48109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/436084720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e48107530 0x7f1e48109980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009ca0 tx=0x7f1e4002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 -- 172.21.15.31:0/436084720 shutdown_connections 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/436084720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e48107530 0x7f1e48109980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/436084720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 0x7f1e48106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/436084720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e481022d0 0x7f1e481046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 -- 172.21.15.31:0/436084720 >> 172.21.15.31:0/436084720 conn(0x7f1e480fbcc0 msgr2=0x7f1e480fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 -- 172.21.15.31:0/436084720 shutdown_connections 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.790+0000 7f1e4cadd640 1 -- 172.21.15.31:0/436084720 wait complete. 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 Processor -- start 2024-04-25T00:31:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 -- start start 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 0x7f1e481978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 0x7f1e48197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e48107530 0x7f1e4819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810d330 con 0x7f1e481022d0 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810d1b0 con 0x7f1e48107530 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4cadd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810d4b0 con 0x7f1e48104c00 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 0x7f1e481978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 0x7f1e481978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44284/0 (socket says 172.21.15.31:44284) 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 -- 172.21.15.31:0/55514639 learned_addr learned my addr 172.21.15.31:0/55514639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e48107530 0x7f1e4819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 0x7f1e48197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 -- 172.21.15.31:0/55514639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 msgr2=0x7f1e48197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 0x7f1e48197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 -- 172.21.15.31:0/55514639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e48107530 msgr2=0x7f1e4819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e48107530 0x7f1e4819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e4659c640 1 -- 172.21.15.31:0/55514639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4819f500 con 0x7f1e481022d0 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.791+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 0x7f1e48197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.792+0000 7f1e4659c640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 0x7f1e481978e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c890 tx=0x7f1e3800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.792+0000 7f1e377fe640 1 -- 172.21.15.31:0/55514639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38004220 con 0x7f1e481022d0 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.792+0000 7f1e377fe640 1 -- 172.21.15.31:0/55514639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e380043c0 con 0x7f1e481022d0 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.792+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4819f7f0 con 0x7f1e481022d0 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.792+0000 7f1e377fe640 1 -- 172.21.15.31:0/55514639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38004a00 con 0x7f1e481022d0 2024-04-25T00:31:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.792+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4819fd90 con 0x7f1e481022d0 2024-04-25T00:31:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e377fe640 1 -- 172.21.15.31:0/55514639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1e38004ba0 con 0x7f1e481022d0 2024-04-25T00:31:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 Processor -- start 2024-04-25T00:31:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 -- start start 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bba9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 0x55a75bbb0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a75bb0ccb0 con 0x55a75bbae9b0 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a75bab55b0 con 0x55a75bba8e50 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.795+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a75bb08c70 con 0x55a75bba9840 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e377fe640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1e300a2cb0 0x7f1e300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 0x55a75bbb0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 0x55a75bbb0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44286/0 (socket says 172.21.15.31:44286) 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 -- 172.21.15.31:0/3645337244 learned_addr learned my addr 172.21.15.31:0/3645337244 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bba9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e4659c640 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1e300a2cb0 0x7f1e300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e377fe640 1 -- 172.21.15.31:0/55514639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f1e380cecb0 con 0x7f1e481022d0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 -- 172.21.15.31:0/3645337244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 msgr2=0x55a75bbae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 -- 172.21.15.31:0/3645337244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 msgr2=0x55a75bba9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bba9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 -- 172.21.15.31:0/3645337244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75b821210 con 0x55a75bbae9b0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e4659c640 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 0x55a75bbb0e40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00bce0 tx=0x7f1e3c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f65c59d47c3d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.796+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1e300a2cb0 0x7f1e300a5170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009430 tx=0x7f1e40005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e34ff9640 1 -- 172.21.15.31:0/3645337244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c015070 con 0x55a75bbae9b0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e34ff9640 1 -- 172.21.15.31:0/3645337244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e3c01a910 con 0x55a75bbae9b0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e34ff9640 1 -- 172.21.15.31:0/3645337244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c011990 con 0x55a75bbae9b0 2024-04-25T00:31:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3645337244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 msgr2=0x55a75bbb0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 0x55a75bbb0e40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00bce0 tx=0x7f1e3c00f730 comp rx=0 tx=0).stop 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3645337244 shutdown_connections 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bbae9b0 0x55a75bbb0e40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3645337244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bba9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3645337244 >> 172.21.15.31:0/3645337244 conn(0x55a75bba44c0 msgr2=0x55a75bba48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.797+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3645337244 shutdown_connections 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3645337244 wait complete. 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 -- start start 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 Processor -- start 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbb3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bbae9b0 0x55a75bbba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a75bbb1960 con 0x55a75bba8e50 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a75bac8b50 con 0x55a75bbae9b0 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a75bbb1ae0 con 0x55a75bba9840 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4f7ea740 10 client.0 _ll_register_callbacks cb 0x55a75bb03ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44300/0 (socket says 172.21.15.31:44300) 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e4659c640 1 -- 172.21.15.31:0/1134652600 learned_addr learned my addr 172.21.15.31:0/1134652600 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbb3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.798+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bbae9b0 0x55a75bbba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4659c640 1 -- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 msgr2=0x55a75bbb3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4659c640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbb3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4659c640 1 -- 172.21.15.31:0/1134652600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bbae9b0 msgr2=0x55a75bbba7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4659c640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bbae9b0 0x55a75bbba7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4659c640 1 -- 172.21.15.31:0/1134652600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75b8ed620 con 0x55a75bba8e50 2024-04-25T00:31:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4659c640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb3250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e380058c0 tx=0x7f1e380053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c733ce1b1779e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e2dffb640 1 -- 172.21.15.31:0/1134652600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809ae40 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e2dffb640 1 -- 172.21.15.31:0/1134652600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f1e380050b0 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c01a910 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e2dffb640 1 -- 172.21.15.31:0/1134652600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e3809bbe0 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4f7ea740 10 client.15477 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a75bbbdc00 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.799+0000 7f1e2dffb640 1 -- 172.21.15.31:0/1134652600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e3c01a910 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e2dffb640 1 -- 172.21.15.31:0/1134652600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1e3801cc40 con 0x55a75bba8e50 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e4f7ea740 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e4f7ea740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e4f7ea740 20 client.15477 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bbbf2d0 0x55a75bbc1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e4f7ea740 4 client.15477 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a75bb1a1f0 con 0x55a75bbbf2d0 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bbbf2d0 0x55a75bbc1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bbbf2d0 0x55a75bbc1720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bed52bc43f120c2 server_cookie=abecaf23d00cca0 in_seq=0 out_seq=0 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.800+0000 7f1e2dffb640 10 client.15477 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e2dffb640 1 -- 172.21.15.31:0/1134652600 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a75bb1a1f0 con 0x55a75bbbf2d0 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e2dffb640 10 client.15477 handle_command_reply: tid=0 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 client.15477 shutdown 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 msgr2=0x55a75bbb3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb3250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e380058c0 tx=0x7f1e380053f0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 shutdown_connections 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bbbf2d0 0x55a75bbc1720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bbae9b0 0x55a75bbba7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bba9840 0x55a75bbb3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/1134652600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb3250 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e2dffb640 0 client.15477 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 >> 172.21.15.31:0/1134652600 conn(0x55a75bba44c0 msgr2=0x55a75bba35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 shutdown_connections 2024-04-25T00:31:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/1134652600 wait complete. 2024-04-25T00:31:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.801+0000 7f1e4f7ea740 20 client.15477 trim_cache size 0 max 16384 2024-04-25T00:31:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 Processor -- start 2024-04-25T00:31:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 -- start start 2024-04-25T00:31:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bbb3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 0x55a75bbba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bba2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481a8b40 con 0x55a75bba9840 2024-04-25T00:31:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481aa680 con 0x55a75bba8e50 2024-04-25T00:31:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481a9210 con 0x55a75bbae9b0 2024-04-25T00:31:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 0x55a75bbba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 0x55a75bbba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44306/0 (socket says 172.21.15.31:44306) 2024-04-25T00:31:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 -- 172.21.15.31:0/2274186483 learned_addr learned my addr 172.21.15.31:0/2274186483 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bba2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 -- 172.21.15.31:0/2274186483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 msgr2=0x55a75bba2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e4659c640 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bbb3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bba2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 -- 172.21.15.31:0/2274186483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 msgr2=0x55a75bbb3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bbb3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e45d9b640 1 -- 172.21.15.31:0/2274186483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75bba4060 con 0x55a75bba9840 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.817+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bba2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 0x55a75bbba760 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55a75bba5430 tx=0x7f1e3c00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9645fd3f430cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e2ffff640 1 -- 172.21.15.31:0/2274186483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c02c070 con 0x55a75bba9840 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e2ffff640 1 -- 172.21.15.31:0/2274186483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e3c005450 con 0x55a75bba9840 2024-04-25T00:31:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e2ffff640 1 -- 172.21.15.31:0/2274186483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c015040 con 0x55a75bba9840 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/2274186483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 msgr2=0x55a75bbba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 0x55a75bbba760 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55a75bba5430 tx=0x7f1e3c00b600 comp rx=0 tx=0).stop 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/2274186483 shutdown_connections 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bba2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba9840 0x55a75bbba760 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/2274186483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba8e50 0x55a75bbb3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/2274186483 >> 172.21.15.31:0/2274186483 conn(0x55a75bba44c0 msgr2=0x55a75bbb2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/2274186483 shutdown_connections 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.818+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/2274186483 wait complete. 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 -- start start 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 Processor -- start 2024-04-25T00:31:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba9840 0x55a75bbb7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bc23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481a9fb0 con 0x55a75bba8e50 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481a98e0 con 0x55a75bba9840 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481a6b20 con 0x55a75bbae9b0 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4f7ea740 10 client.0 _ll_register_callbacks cb 0x7f1e380970c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44318/0 (socket says 172.21.15.31:44318) 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 -- 172.21.15.31:0/3618880445 learned_addr learned my addr 172.21.15.31:0/3618880445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba9840 0x55a75bbb7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e46d9d640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bc23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 -- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 msgr2=0x55a75bc23890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bc23890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 -- 172.21.15.31:0/3618880445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba9840 msgr2=0x55a75bbb7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba9840 0x55a75bbb7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.819+0000 7f1e4659c640 1 -- 172.21.15.31:0/3618880445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75bbb8100 con 0x55a75bba8e50 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4659c640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb7580 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ccaf0 tx=0x7f1e3800a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ce7153dfabc5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e2dffb640 1 -- 172.21.15.31:0/3618880445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809f890 con 0x55a75bba8e50 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e2dffb640 1 -- 172.21.15.31:0/3618880445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f1e380972d0 con 0x55a75bba8e50 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c005450 con 0x55a75bba8e50 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e2dffb640 1 -- 172.21.15.31:0/3618880445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e380a4150 con 0x55a75bba8e50 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 10 client.15483 fetch_fsmap learned FSMap version 26 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a75bc24e50 con 0x55a75bba8e50 2024-04-25T00:31:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e2dffb640 1 -- 172.21.15.31:0/3618880445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e3c005450 con 0x55a75bba8e50 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e2dffb640 1 -- 172.21.15.31:0/3618880445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1e3809a5a0 con 0x55a75bba8e50 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 20 client.15483 populate_metadata read hostname 'smithi031' 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bc25610 0x55a75bc27a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 4 client.15483 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.820+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a75bb14540 con 0x55a75bc25610 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.821+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bc25610 0x55a75bc27a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.821+0000 7f1e2dffb640 10 client.15483 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.821+0000 7f1e45d9b640 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bc25610 0x55a75bc27a00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95967650bc0c0b66 server_cookie=d78cf9067231bcf8 in_seq=0 out_seq=0 2024-04-25T00:31:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e2dffb640 1 -- 172.21.15.31:0/3618880445 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a75bb14540 con 0x55a75bc25610 2024-04-25T00:31:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e2dffb640 10 client.15483 handle_command_reply: tid=0 2024-04-25T00:31:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 client.15483 shutdown 2024-04-25T00:31:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 msgr2=0x55a75bbb7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb7580 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ccaf0 tx=0x7f1e3800a520 comp rx=0 tx=0).stop 2024-04-25T00:31:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 shutdown_connections 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a75bc25610 0x55a75bc27a00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bbae9b0 0x55a75bc23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a75bba9840 0x55a75bbb7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 --2- 172.21.15.31:0/3618880445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a75bba8e50 0x55a75bbb7580 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e2dffb640 0 client.15483 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 >> 172.21.15.31:0/3618880445 conn(0x55a75bba44c0 msgr2=0x55a75bba2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 shutdown_connections 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 1 -- 172.21.15.31:0/3618880445 wait complete. 2024-04-25T00:31:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.832+0000 7f1e4f7ea740 20 client.15483 trim_cache size 0 max 16384 2024-04-25T00:31:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:31:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:31:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:31:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1e300a2cb0 msgr2=0x7f1e300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1e300a2cb0 0x7f1e300a5170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009430 tx=0x7f1e40005d80 comp rx=0 tx=0).stop 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 msgr2=0x7f1e481978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 0x7f1e481978e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c890 tx=0x7f1e3800cd50 comp rx=0 tx=0).stop 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 shutdown_connections 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e48107530 0x7f1e4819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e48104c00 0x7f1e48197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f1e300a2cb0 0x7f1e300a5170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 --2- 172.21.15.31:0/55514639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e481022d0 0x7f1e481978e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 >> 172.21.15.31:0/55514639 conn(0x7f1e480fbcc0 msgr2=0x7f1e48102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 shutdown_connections 2024-04-25T00:31:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:31:48.833+0000 7f1e4cadd640 1 -- 172.21.15.31:0/55514639 wait complete. 2024-04-25T00:31:49.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:48 smithi169 ceph-mon[33112]: pgmap v398: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:49.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:48 smithi040 ceph-mon[32945]: pgmap v398: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:49.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:48 smithi031 ceph-mon[29167]: pgmap v398: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:50 smithi169 ceph-mon[33112]: pgmap v399: 97 pgs: 97 active+clean; 671 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T00:31:51.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:50 smithi040 ceph-mon[32945]: pgmap v399: 97 pgs: 97 active+clean; 671 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T00:31:51.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:50 smithi031 ceph-mon[29167]: pgmap v399: 97 pgs: 97 active+clean; 671 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T00:31:53.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:52 smithi169 ceph-mon[33112]: pgmap v400: 97 pgs: 97 active+clean; 671 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:52 smithi040 ceph-mon[32945]: pgmap v400: 97 pgs: 97 active+clean; 671 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:52 smithi031 ceph-mon[29167]: pgmap v400: 97 pgs: 97 active+clean; 671 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T00:31:55.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:54 smithi169 ceph-mon[33112]: pgmap v401: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:31:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:54 smithi040 ceph-mon[32945]: pgmap v401: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:31:55.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:54 smithi031 ceph-mon[29167]: pgmap v401: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:31:57.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:56 smithi169 ceph-mon[33112]: pgmap v402: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-04-25T00:31:57.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:56 smithi040 ceph-mon[32945]: pgmap v402: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-04-25T00:31:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:56 smithi031 ceph-mon[29167]: pgmap v402: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-04-25T00:31:59.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:31:58 smithi169 ceph-mon[33112]: pgmap v403: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-25T00:31:59.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:31:58 smithi040 ceph-mon[32945]: pgmap v403: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-25T00:31:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:31:58 smithi031 ceph-mon[29167]: pgmap v403: 97 pgs: 97 active+clean; 723 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-25T00:32:01.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:00 smithi169 ceph-mon[33112]: pgmap v404: 97 pgs: 97 active+clean; 781 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-04-25T00:32:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:00 smithi040 ceph-mon[32945]: pgmap v404: 97 pgs: 97 active+clean; 781 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-04-25T00:32:01.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:00 smithi031 ceph-mon[29167]: pgmap v404: 97 pgs: 97 active+clean; 781 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-04-25T00:32:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 -- 172.21.15.31:0/1250076553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 msgr2=0x7f62f8106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/1250076553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8106fa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009900 tx=0x7f62ec02a410 comp rx=0 tx=0).stop 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 -- 172.21.15.31:0/1250076553 shutdown_connections 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/1250076553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81074e0 0x7f62f81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/1250076553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8106fa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/1250076553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 0x7f62f8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 -- 172.21.15.31:0/1250076553 >> 172.21.15.31:0/1250076553 conn(0x7f62f80fbc10 msgr2=0x7f62f80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 -- 172.21.15.31:0/1250076553 shutdown_connections 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 -- 172.21.15.31:0/1250076553 wait complete. 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 Processor -- start 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.934+0000 7f62fe2ca640 1 -- start start 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fe2ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 0x7f62f8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fe2ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fe2ca640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81074e0 0x7f62f819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fe2ca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f62f810d320 con 0x7f62f8104bb0 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fe2ca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f62f810d1a0 con 0x7f62f8102280 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fe2ca640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62f810d4a0 con 0x7f62f81074e0 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47856/0 (socket says 172.21.15.31:47856) 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 -- 172.21.15.31:0/2484900469 learned_addr learned my addr 172.21.15.31:0/2484900469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f7fff640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 0x7f62f8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62fc867640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81074e0 0x7f62f819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 -- 172.21.15.31:0/2484900469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81074e0 msgr2=0x7f62f819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81074e0 0x7f62f819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 -- 172.21.15.31:0/2484900469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 msgr2=0x7f62f8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 0x7f62f8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f77fe640 1 -- 172.21.15.31:0/2484900469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f819f590 con 0x7f62f8104bb0 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.935+0000 7f62f7fff640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 0x7f62f8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.936+0000 7f62f77fe640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8197eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0314e0 tx=0x7f62ec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.936+0000 7f62f57fa640 1 -- 172.21.15.31:0/2484900469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec03f070 con 0x7f62f8104bb0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.936+0000 7f62f57fa640 1 -- 172.21.15.31:0/2484900469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62ec031c40 con 0x7f62f8104bb0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.936+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f819f820 con 0x7f62f8104bb0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.936+0000 7f62f57fa640 1 -- 172.21.15.31:0/2484900469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec035030 con 0x7f62f8104bb0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.936+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f819fd10 con 0x7f62f8104bb0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.938+0000 7f62f57fa640 1 -- 172.21.15.31:0/2484900469 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f62ec042070 con 0x7f62f8104bb0 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.938+0000 7f6300fd7740 1 Processor -- start 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 -- start start 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b00e50 0x557037b01270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 0x557037b06340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b08e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557037a64cb0 con 0x557037b01840 2024-04-25T00:32:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557037a0d5b0 con 0x557037b00e50 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f6300fd7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557037a60c70 con 0x557037b069b0 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 0x557037b06340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 0x557037b06340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47860/0 (socket says 172.21.15.31:47860) 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 -- 172.21.15.31:0/391399464 learned_addr learned my addr 172.21.15.31:0/391399464 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f57fa640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62e00a2c30 0x7f62e00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f77fe640 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b08e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62fc867640 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b00e50 0x557037b01270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 -- 172.21.15.31:0/391399464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 msgr2=0x557037b08e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b08e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 -- 172.21.15.31:0/391399464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b00e50 msgr2=0x557037b01270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b00e50 0x557037b01270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f7fff640 1 -- 172.21.15.31:0/391399464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557037779210 con 0x557037b01840 2024-04-25T00:32:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62fc867640 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b00e50 0x557037b01270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62fc867640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62e00a2c30 0x7f62e00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.939+0000 7f62f57fa640 1 -- 172.21.15.31:0/2484900469 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f62ec0ef800 con 0x7f62f8104bb0 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f62f7fff640 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 0x557037b06340 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f62e8010580 tx=0x7f62e8010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc090ec99d4f3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f62de7fc640 1 -- 172.21.15.31:0/391399464 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8011760 con 0x557037b01840 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f62de7fc640 1 -- 172.21.15.31:0/391399464 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e8011db0 con 0x557037b01840 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f62de7fc640 1 -- 172.21.15.31:0/391399464 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e800ee00 con 0x557037b01840 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f6300fd7740 1 -- 172.21.15.31:0/391399464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 msgr2=0x557037b06340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f62fc867640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62e00a2c30 0x7f62e00a50f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x557037b020a0 tx=0x7f62f0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f6300fd7740 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 0x557037b06340 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f62e8010580 tx=0x7f62e8010a40 comp rx=0 tx=0).stop 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f6300fd7740 1 -- 172.21.15.31:0/391399464 shutdown_connections 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f6300fd7740 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b08e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.940+0000 7f6300fd7740 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b01840 0x557037b06340 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 --2- 172.21.15.31:0/391399464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b00e50 0x557037b01270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- 172.21.15.31:0/391399464 >> 172.21.15.31:0/391399464 conn(0x557037afc4c0 msgr2=0x557037afc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- 172.21.15.31:0/391399464 shutdown_connections 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- 172.21.15.31:0/391399464 wait complete. 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- start start 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 Processor -- start 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037b0b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557037b09960 con 0x557037b00e50 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557037a20b50 con 0x557037b01840 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.941+0000 7f6300fd7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557037b09ae0 con 0x557037b069b0 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f6300fd7740 10 client.0 _ll_register_callbacks cb 0x557037a5bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47876/0 (socket says 172.21.15.31:47876) 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 -- 172.21.15.31:0/1787160395 learned_addr learned my addr 172.21.15.31:0/1787160395 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f77fe640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037b0b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62fc867640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 -- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 msgr2=0x557037b128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 -- 172.21.15.31:0/1787160395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 msgr2=0x557037b0b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037b0b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 -- 172.21.15.31:0/1787160395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557037845620 con 0x557037b00e50 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f77fe640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037b0b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62f7fff640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0b330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005d00 tx=0x7f62e800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cafc0d01aa38fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62d77fe640 1 -- 172.21.15.31:0/1787160395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e802b060 con 0x557037b00e50 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62d77fe640 1 -- 172.21.15.31:0/1787160395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f62e800e740 con 0x557037b00e50 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e8011db0 con 0x557037b00e50 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f62d77fe640 1 -- 172.21.15.31:0/1787160395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62e8011560 con 0x557037b00e50 2024-04-25T00:32:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f6300fd7740 10 client.15495 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.942+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557037b15c00 con 0x557037b00e50 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f62d77fe640 1 -- 172.21.15.31:0/1787160395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e8011db0 con 0x557037b00e50 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f62d77fe640 1 -- 172.21.15.31:0/1787160395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f62e801ca90 con 0x557037b00e50 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f6300fd7740 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f6300fd7740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f6300fd7740 20 client.15495 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f6300fd7740 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b17000 0x557037b193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f6300fd7740 4 client.15495 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557037a721f0 con 0x557037b17000 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.943+0000 7f62f77fe640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b17000 0x557037b193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.944+0000 7f62f77fe640 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b17000 0x557037b193f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a7e07baa68bce6 server_cookie=a7ea5e22f7edd88c in_seq=0 out_seq=0 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.944+0000 7f62d77fe640 10 client.15495 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.944+0000 7f62d77fe640 1 -- 172.21.15.31:0/1787160395 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557037a721f0 con 0x557037b17000 2024-04-25T00:32:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.944+0000 7f62d77fe640 10 client.15495 handle_command_reply: tid=0 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.944+0000 7f6300fd7740 1 client.15495 shutdown 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 msgr2=0x557037b0b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0b330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005d00 tx=0x7f62e800eef0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 shutdown_connections 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b17000 0x557037b193f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b069b0 0x557037b128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037b0b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f62d77fe640 0 client.15495 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 --2- 172.21.15.31:0/1787160395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0b330 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 >> 172.21.15.31:0/1787160395 conn(0x557037afc4c0 msgr2=0x557037afb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 shutdown_connections 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 1 -- 172.21.15.31:0/1787160395 wait complete. 2024-04-25T00:32:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.945+0000 7f6300fd7740 20 client.15495 trim_cache size 0 max 16384 2024-04-25T00:32:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 Processor -- start 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 -- start start 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b00e50 0x557037aff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037affe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 0x557037b12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81a8c20 con 0x557037b069b0 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81aa760 con 0x557037b01840 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f6300fd7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81a92f0 con 0x557037b00e50 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 0x557037b12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 0x557037b12870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47886/0 (socket says 172.21.15.31:47886) 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 -- 172.21.15.31:0/2424334565 learned_addr learned my addr 172.21.15.31:0/2424334565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62f7fff640 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b00e50 0x557037aff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62f77fe640 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037affe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 -- 172.21.15.31:0/2424334565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b00e50 msgr2=0x557037aff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b00e50 0x557037aff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 -- 172.21.15.31:0/2424334565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 msgr2=0x557037affe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037affe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62fc867640 1 -- 172.21.15.31:0/2424334565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557037afc060 con 0x557037b069b0 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.960+0000 7f62f7fff640 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b00e50 0x557037aff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f62fc867640 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 0x557037b12870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f62f0003e20 tx=0x7f62f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7cd66c93d1cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f62ddffb640 1 -- 172.21.15.31:0/2424334565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0029440 con 0x557037b069b0 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f62ddffb640 1 -- 172.21.15.31:0/2424334565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62f0029a90 con 0x557037b069b0 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f62ddffb640 1 -- 172.21.15.31:0/2424334565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f001fae0 con 0x557037b069b0 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 -- 172.21.15.31:0/2424334565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 msgr2=0x557037b12870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 0x557037b12870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f62f0003e20 tx=0x7f62f0016000 comp rx=0 tx=0).stop 2024-04-25T00:32:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 -- 172.21.15.31:0/2424334565 shutdown_connections 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b069b0 0x557037b12870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b01840 0x557037affe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 --2- 172.21.15.31:0/2424334565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b00e50 0x557037aff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 -- 172.21.15.31:0/2424334565 >> 172.21.15.31:0/2424334565 conn(0x557037afc4c0 msgr2=0x557037aff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 -- 172.21.15.31:0/2424334565 shutdown_connections 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.961+0000 7f6300fd7740 1 -- 172.21.15.31:0/2424334565 wait complete. 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 -- start start 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 Processor -- start 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b01840 0x557037b0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b069b0 0x557037b10f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81aa090 con 0x557037b00e50 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81a99c0 con 0x557037b069b0 2024-04-25T00:32:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81a6c00 con 0x557037b01840 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f6300fd7740 10 client.0 _ll_register_callbacks cb 0x7f62e80103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47900/0 (socket says 172.21.15.31:47900) 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f62f7fff640 1 -- 172.21.15.31:0/3782364872 learned_addr learned my addr 172.21.15.31:0/3782364872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.962+0000 7f62fc867640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b069b0 0x557037b10f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f77fe640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b01840 0x557037b0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f7fff640 1 -- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b01840 msgr2=0x557037b0ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f7fff640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b01840 0x557037b0ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f7fff640 1 -- 172.21.15.31:0/3782364872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b069b0 msgr2=0x557037b10f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f7fff640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b069b0 0x557037b10f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f7fff640 1 -- 172.21.15.31:0/3782364872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557037b11550 con 0x557037b00e50 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f77fe640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b01840 0x557037b0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62f7fff640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0fa60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f62e80059f0 tx=0x7f62e800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e9f50eac09479c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62d77fe640 1 -- 172.21.15.31:0/3782364872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e802b070 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62d77fe640 1 -- 172.21.15.31:0/3782364872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f62e8031b10 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f0029a90 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f62d77fe640 1 -- 172.21.15.31:0/3782364872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62e8040030 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f6300fd7740 10 client.15501 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.963+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557037b127d0 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f62d77fe640 1 -- 172.21.15.31:0/3782364872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62f0029a90 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f62d77fe640 1 -- 172.21.15.31:0/3782364872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f62e8038020 con 0x557037b00e50 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f6300fd7740 10 client.15501 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f6300fd7740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f6300fd7740 20 client.15501 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f6300fd7740 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b7d840 0x557037b7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f6300fd7740 4 client.15501 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557037b027b0 con 0x557037b7d840 2024-04-25T00:32:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f62f77fe640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b7d840 0x557037b7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f62f77fe640 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b7d840 0x557037b7fc30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25c1e891243a255 server_cookie=f9adbd781430feee in_seq=0 out_seq=0 2024-04-25T00:32:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.964+0000 7f62d77fe640 10 client.15501 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f62d77fe640 1 -- 172.21.15.31:0/3782364872 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557037b027b0 con 0x557037b7d840 2024-04-25T00:32:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f62d77fe640 10 client.15501 handle_command_reply: tid=0 2024-04-25T00:32:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 client.15501 shutdown 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 msgr2=0x557037b0fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0fa60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f62e80059f0 tx=0x7f62e800bd20 comp rx=0 tx=0).stop 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 shutdown_connections 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557037b7d840 0x557037b7fc30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557037b069b0 0x557037b10f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557037b01840 0x557037b0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f62d77fe640 0 client.15501 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 --2- 172.21.15.31:0/3782364872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557037b00e50 0x557037b0fa60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 >> 172.21.15.31:0/3782364872 conn(0x557037afc4c0 msgr2=0x557037affad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 shutdown_connections 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 1 -- 172.21.15.31:0/3782364872 wait complete. 2024-04-25T00:32:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.978+0000 7f6300fd7740 20 client.15501 trim_cache size 0 max 16384 2024-04-25T00:32:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:32:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:32:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:32:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62e00a2c30 msgr2=0x7f62e00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62e00a2c30 0x7f62e00a50f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x557037b020a0 tx=0x7f62f0004a40 comp rx=0 tx=0).stop 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 msgr2=0x7f62f8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8197eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0314e0 tx=0x7f62ec003c10 comp rx=0 tx=0).stop 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 shutdown_connections 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81074e0 0x7f62f819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f62e00a2c30 0x7f62e00a50f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62f8104bb0 0x7f62f8197eb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 --2- 172.21.15.31:0/2484900469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62f8102280 0x7f62f8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 >> 172.21.15.31:0/2484900469 conn(0x7f62f80fbc10 msgr2=0x7f62f8102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 shutdown_connections 2024-04-25T00:32:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:01.979+0000 7f62fe2ca640 1 -- 172.21.15.31:0/2484900469 wait complete. 2024-04-25T00:32:03.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:02 smithi169 ceph-mon[33112]: pgmap v405: 97 pgs: 97 active+clean; 781 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:32:03.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:02 smithi040 ceph-mon[32945]: pgmap v405: 97 pgs: 97 active+clean; 781 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:32:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:02 smithi031 ceph-mon[29167]: pgmap v405: 97 pgs: 97 active+clean; 781 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:32:05.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:04 smithi031 ceph-mon[29167]: pgmap v406: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 23 op/s 2024-04-25T00:32:05.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:04 smithi169 ceph-mon[33112]: pgmap v406: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 23 op/s 2024-04-25T00:32:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:04 smithi040 ceph-mon[32945]: pgmap v406: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 23 op/s 2024-04-25T00:32:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:06 smithi031 ceph-mon[29167]: pgmap v407: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:07.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:06 smithi169 ceph-mon[33112]: pgmap v407: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:06 smithi040 ceph-mon[32945]: pgmap v407: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:08 smithi031 ceph-mon[29167]: pgmap v408: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:09.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:08 smithi169 ceph-mon[33112]: pgmap v408: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:08 smithi040 ceph-mon[32945]: pgmap v408: 97 pgs: 97 active+clean; 843 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:10 smithi031 ceph-mon[29167]: pgmap v409: 97 pgs: 97 active+clean; 897 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:32:11.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:10 smithi169 ceph-mon[33112]: pgmap v409: 97 pgs: 97 active+clean; 897 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:32:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:10 smithi040 ceph-mon[32945]: pgmap v409: 97 pgs: 97 active+clean; 897 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:32:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:12 smithi031 ceph-mon[29167]: pgmap v410: 97 pgs: 97 active+clean; 897 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:12 smithi169 ceph-mon[33112]: pgmap v410: 97 pgs: 97 active+clean; 897 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:12 smithi040 ceph-mon[32945]: pgmap v410: 97 pgs: 97 active+clean; 897 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:14.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:14 smithi031 ceph-mon[29167]: pgmap v411: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:14.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:14 smithi169 ceph-mon[33112]: pgmap v411: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:14.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:14 smithi040 ceph-mon[32945]: pgmap v411: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 -- 172.21.15.31:0/851304313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a0103380 msgr2=0x7f56a0109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/851304313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a0103380 0x7f56a0109c10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ca0 tx=0x7f569802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 -- 172.21.15.31:0/851304313 shutdown_connections 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/851304313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a0103380 0x7f56a0109c10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/851304313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a01029c0 0x7f56a0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/851304313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a01017d0 0x7f56a0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 -- 172.21.15.31:0/851304313 >> 172.21.15.31:0/851304313 conn(0x7f56a00fcd60 msgr2=0x7f56a00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 -- 172.21.15.31:0/851304313 shutdown_connections 2024-04-25T00:32:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.078+0000 7f56a4cfc640 1 -- 172.21.15.31:0/851304313 wait complete. 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 Processor -- start 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 -- start start 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 0x7f56a01978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a01029c0 0x7f56a0197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a0103380 0x7f56a019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d350 con 0x7f56a01017d0 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d1d0 con 0x7f56a0103380 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f56a4cfc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d4d0 con 0x7f56a01029c0 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 0x7f56a01978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 0x7f56a01978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35930/0 (socket says 172.21.15.31:35930) 2024-04-25T00:32:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.079+0000 7f569e59c640 1 -- 172.21.15.31:0/952560283 learned_addr learned my addr 172.21.15.31:0/952560283 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f5695d9b640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a01029c0 0x7f56a0197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569ed9d640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a0103380 0x7f56a019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569e59c640 1 -- 172.21.15.31:0/952560283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a01029c0 msgr2=0x7f56a0197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569e59c640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a01029c0 0x7f56a0197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569e59c640 1 -- 172.21.15.31:0/952560283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a0103380 msgr2=0x7f56a019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569e59c640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a0103380 0x7f56a019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569e59c640 1 -- 172.21.15.31:0/952560283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019f500 con 0x7f56a01017d0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f569e59c640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 0x7f56a01978e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9c0 tx=0x7f569000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f5697fff640 1 -- 172.21.15.31:0/952560283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569000cd80 con 0x7f56a01017d0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019f7f0 con 0x7f56a01017d0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f5697fff640 1 -- 172.21.15.31:0/952560283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56900035f0 con 0x7f56a01017d0 2024-04-25T00:32:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f5697fff640 1 -- 172.21.15.31:0/952560283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690010690 con 0x7f56a01017d0 2024-04-25T00:32:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.080+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a019fd40 con 0x7f56a01017d0 2024-04-25T00:32:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f5697fff640 1 -- 172.21.15.31:0/952560283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f56900037a0 con 0x7f56a01017d0 2024-04-25T00:32:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 Processor -- start 2024-04-25T00:32:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 -- start start 2024-04-25T00:32:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621c4202cb0 con 0x5621c42a3260 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621c41ab5b0 con 0x5621c42a8dc0 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.083+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621c41fec70 con 0x5621c42a3c50 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5697fff640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f568c0a2c30 0x7f568c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f569ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f569e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f569ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35940/0 (socket says 172.21.15.31:35940) 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f569ed9d640 1 -- 172.21.15.31:0/1040290417 learned_addr learned my addr 172.21.15.31:0/1040290417 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f569ed9d640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f568c0a2c30 0x7f568c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5697fff640 1 -- 172.21.15.31:0/952560283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f56900cf650 con 0x7f56a01017d0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 -- 172.21.15.31:0/1040290417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 msgr2=0x5621c42a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 -- 172.21.15.31:0/1040290417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 msgr2=0x5621c42a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 -- 172.21.15.31:0/1040290417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c3f17210 con 0x5621c42a8dc0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f569ed9d640 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.084+0000 7f5695d9b640 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42ab250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f568800bbc0 tx=0x7f56880107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867dffe15a603b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f5694d99640 1 -- 172.21.15.31:0/1040290417 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688004030 con 0x5621c42a8dc0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f569ed9d640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f568c0a2c30 0x7f568c0a50f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5621c42a44b0 tx=0x7f569802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f5694d99640 1 -- 172.21.15.31:0/1040290417 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56880035f0 con 0x5621c42a8dc0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f5694d99640 1 -- 172.21.15.31:0/1040290417 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880116a0 con 0x5621c42a8dc0 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1040290417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 msgr2=0x5621c42ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42ab250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f568800bbc0 tx=0x7f56880107a0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1040290417 shutdown_connections 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42ab250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1040290417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.085+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1040290417 >> 172.21.15.31:0/1040290417 conn(0x5621c429a4c0 msgr2=0x5621c429a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1040290417 shutdown_connections 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1040290417 wait complete. 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 -- start start 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 Processor -- start 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621c42abd70 con 0x5621c42a3260 2024-04-25T00:32:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621c41beb50 con 0x5621c42a8dc0 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621c42abef0 con 0x5621c42a3c50 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f56a7a0a740 10 client.0 _ll_register_callbacks cb 0x5621c41f9ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35948/0 (socket says 172.21.15.31:35948) 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f569e59c640 1 -- 172.21.15.31:0/1961173498 learned_addr learned my addr 172.21.15.31:0/1961173498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f569ed9d640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.086+0000 7f5695d9b640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f569e59c640 1 -- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 msgr2=0x5621c42adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f569e59c640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f569e59c640 1 -- 172.21.15.31:0/1961173498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 msgr2=0x5621c42b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f569e59c640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f569e59c640 1 -- 172.21.15.31:0/1961173498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c3fe3620 con 0x5621c42a3260 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f569e59c640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42ad650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5690021000 tx=0x7f569006ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c6a5ba123d920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f5685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f5685ffb640 1 -- 172.21.15.31:0/1961173498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569009b4d0 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f5685ffb640 1 -- 172.21.15.31:0/1961173498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f56900cfb00 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56880035f0 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f5685ffb640 1 -- 172.21.15.31:0/1961173498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f569009c580 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f56a7a0a740 10 client.15513 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621c42b7fb0 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.087+0000 7f5685ffb640 1 -- 172.21.15.31:0/1961173498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56880035f0 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f5685ffb640 1 -- 172.21.15.31:0/1961173498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f56900cf110 con 0x5621c42a3260 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f56a7a0a740 10 client.15513 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f56a7a0a740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f56a7a0a740 20 client.15513 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c42b9670 0x5621c42bbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f56a7a0a740 4 client.15513 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c42101f0 con 0x5621c42b9670 2024-04-25T00:32:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.088+0000 7f5695d9b640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c42b9670 0x5621c42bbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.089+0000 7f5695d9b640 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c42b9670 0x5621c42bbac0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db3e40f23b43b1a server_cookie=20f5070de9709781 in_seq=0 out_seq=0 2024-04-25T00:32:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.089+0000 7f5685ffb640 10 client.15513 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.089+0000 7f5685ffb640 1 -- 172.21.15.31:0/1961173498 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621c42101f0 con 0x5621c42b9670 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.089+0000 7f5685ffb640 10 client.15513 handle_command_reply: tid=0 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.089+0000 7f56a7a0a740 1 client.15513 shutdown 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 msgr2=0x5621c42ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42ad650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5690021000 tx=0x7f569006ae00 comp rx=0 tx=0).stop 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 shutdown_connections 2024-04-25T00:32:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c42b9670 0x5621c42bbac0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a8dc0 0x5621c42b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a3c50 0x5621c42adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/1961173498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42ad650 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f5685ffb640 0 client.15513 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 >> 172.21.15.31:0/1961173498 conn(0x5621c429a4c0 msgr2=0x5621c42995c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 shutdown_connections 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 1 -- 172.21.15.31:0/1961173498 wait complete. 2024-04-25T00:32:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.090+0000 7f56a7a0a740 20 client.15513 trim_cache size 0 max 16384 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 Processor -- start 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 -- start start 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c4299440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a69d0 con 0x5621c42a3260 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a8510 con 0x5621c42a3c50 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a70a0 con 0x5621c42a8dc0 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35958/0 (socket says 172.21.15.31:35958) 2024-04-25T00:32:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 -- 172.21.15.31:0/2750061407 learned_addr learned my addr 172.21.15.31:0/2750061407 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569ed9d640 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c4299440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f5695d9b640 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 -- 172.21.15.31:0/2750061407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 msgr2=0x5621c4299440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c4299440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 -- 172.21.15.31:0/2750061407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 msgr2=0x5621c42b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f569e59c640 1 -- 172.21.15.31:0/2750061407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c429a060 con 0x5621c42a3260 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.105+0000 7f5695d9b640 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f569e59c640 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42adaf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f56900cd490 tx=0x7f5690003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba51fe52d0dac94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f5687fff640 1 -- 172.21.15.31:0/2750061407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569009bc50 con 0x5621c42a3260 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f5687fff640 1 -- 172.21.15.31:0/2750061407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56900a4af0 con 0x5621c42a3260 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f5687fff640 1 -- 172.21.15.31:0/2750061407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569009b530 con 0x5621c42a3260 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2750061407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 msgr2=0x5621c42adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42adaf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f56900cd490 tx=0x7f5690003b90 comp rx=0 tx=0).stop 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2750061407 shutdown_connections 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c4299440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2750061407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42adaf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2750061407 >> 172.21.15.31:0/2750061407 conn(0x5621c429a4c0 msgr2=0x5621c4298d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2750061407 shutdown_connections 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.106+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2750061407 wait complete. 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 -- start start 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 Processor -- start 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c42b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a7e40 con 0x5621c42a3260 2024-04-25T00:32:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a7770 con 0x5621c42a3c50 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a49b0 con 0x5621c42a8dc0 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f56a7a0a740 10 client.0 _ll_register_callbacks cb 0x7f5690003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35966/0 (socket says 172.21.15.31:35966) 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 -- 172.21.15.31:0/2535540093 learned_addr learned my addr 172.21.15.31:0/2535540093 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f5695d9b640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569ed9d640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c42b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 -- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 msgr2=0x5621c42b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c42b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 -- 172.21.15.31:0/2535540093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 msgr2=0x5621c42b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.107+0000 7f569e59c640 1 -- 172.21.15.31:0/2535540093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c42b4560 con 0x5621c42a3260 2024-04-25T00:32:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f569e59c640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42b2a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f569000cfd0 tx=0x7f569009b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57ab3e3df88c550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f5685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f5685ffb640 1 -- 172.21.15.31:0/2535540093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569009bc50 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f5685ffb640 1 -- 172.21.15.31:0/2535540093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f56900baac0 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56900a4af0 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f5685ffb640 1 -- 172.21.15.31:0/2535540093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f56900a0230 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 10 client.15519 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621c4320230 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f5685ffb640 1 -- 172.21.15.31:0/2535540093 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56900a4af0 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f5685ffb640 1 -- 172.21.15.31:0/2535540093 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f56900a1550 con 0x5621c42a3260 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 10 client.15519 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 20 client.15519 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c4320930 0x5621c4322d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 4 client.15519 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.108+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621c420a540 con 0x5621c4320930 2024-04-25T00:32:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.109+0000 7f5695d9b640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c4320930 0x5621c4322d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.109+0000 7f5695d9b640 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c4320930 0x5621c4322d20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8989be353e4f2be server_cookie=a4b54961e0f6d0d2 in_seq=0 out_seq=0 2024-04-25T00:32:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.109+0000 7f5685ffb640 10 client.15519 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f5685ffb640 1 -- 172.21.15.31:0/2535540093 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621c420a540 con 0x5621c4320930 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f5685ffb640 10 client.15519 handle_command_reply: tid=0 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 client.15519 shutdown 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 msgr2=0x5621c42b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42b2a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f569000cfd0 tx=0x7f569009b4e0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 shutdown_connections 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5621c4320930 0x5621c4322d20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621c42a8dc0 0x5621c42b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f5685ffb640 0 client.15519 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621c42a3c50 0x5621c42b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 --2- 172.21.15.31:0/2535540093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621c42a3260 0x5621c42b2a20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 >> 172.21.15.31:0/2535540093 conn(0x5621c429a4c0 msgr2=0x5621c42995c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 shutdown_connections 2024-04-25T00:32:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.121+0000 7f56a7a0a740 1 -- 172.21.15.31:0/2535540093 wait complete. 2024-04-25T00:32:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a7a0a740 20 client.15519 trim_cache size 0 max 16384 2024-04-25T00:32:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:32:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:32:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:32:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f568c0a2c30 msgr2=0x7f568c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f568c0a2c30 0x7f568c0a50f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5621c42a44b0 tx=0x7f569802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 msgr2=0x7f56a01978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 0x7f56a01978e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9c0 tx=0x7f569000ee80 comp rx=0 tx=0).stop 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 shutdown_connections 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a0103380 0x7f56a019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a01029c0 0x7f56a0197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f568c0a2c30 0x7f568c0a50f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 --2- 172.21.15.31:0/952560283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a01017d0 0x7f56a01978e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 >> 172.21.15.31:0/952560283 conn(0x7f56a00fcd60 msgr2=0x7f56a0105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 shutdown_connections 2024-04-25T00:32:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:15.122+0000 7f56a4cfc640 1 -- 172.21.15.31:0/952560283 wait complete. 2024-04-25T00:32:16.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:16 smithi031 ceph-mon[29167]: pgmap v412: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:32:16.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:16 smithi169 ceph-mon[33112]: pgmap v412: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:32:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:16 smithi040 ceph-mon[32945]: pgmap v412: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:32:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:18 smithi031 ceph-mon[29167]: pgmap v413: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:32:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:18 smithi169 ceph-mon[33112]: pgmap v413: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:32:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:18 smithi040 ceph-mon[32945]: pgmap v413: 97 pgs: 97 active+clean; 953 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:32:20.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:20 smithi031 ceph-mon[29167]: pgmap v414: 97 pgs: 97 active+clean; 1018 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:32:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:20 smithi169 ceph-mon[33112]: pgmap v414: 97 pgs: 97 active+clean; 1018 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:32:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:20 smithi040 ceph-mon[32945]: pgmap v414: 97 pgs: 97 active+clean; 1018 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:32:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:22 smithi031 ceph-mon[29167]: pgmap v415: 97 pgs: 97 active+clean; 1018 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:32:22.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:22 smithi169 ceph-mon[33112]: pgmap v415: 97 pgs: 97 active+clean; 1018 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:32:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:22 smithi040 ceph-mon[32945]: pgmap v415: 97 pgs: 97 active+clean; 1018 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:32:24.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:24 smithi169 ceph-mon[33112]: pgmap v416: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:24 smithi031 ceph-mon[29167]: pgmap v416: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:24 smithi040 ceph-mon[32945]: pgmap v416: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:26.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:26 smithi040 ceph-mon[32945]: pgmap v417: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:26 smithi031 ceph-mon[29167]: pgmap v417: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:26 smithi169 ceph-mon[33112]: pgmap v417: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.221+0000 7f77aa2bd640 1 -- 172.21.15.31:0/1889997932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a41029c0 msgr2=0x7f77a4102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.221+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/1889997932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a41029c0 0x7f77a4102e40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f779402a410 comp rx=0 tx=0).stop 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.221+0000 7f77aa2bd640 1 -- 172.21.15.31:0/1889997932 shutdown_connections 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.221+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/1889997932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a4103380 0x7f77a4109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.221+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/1889997932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a41029c0 0x7f77a4102e40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.221+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/1889997932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41017d0 0x7f77a4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.222+0000 7f77aa2bd640 1 -- 172.21.15.31:0/1889997932 >> 172.21.15.31:0/1889997932 conn(0x7f77a40fcd60 msgr2=0x7f77a40ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.222+0000 7f77aa2bd640 1 -- 172.21.15.31:0/1889997932 shutdown_connections 2024-04-25T00:32:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.222+0000 7f77aa2bd640 1 -- 172.21.15.31:0/1889997932 wait complete. 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.222+0000 7f77aa2bd640 1 Processor -- start 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.222+0000 7f77aa2bd640 1 -- start start 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77aa2bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41017d0 0x7f77a4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77aa2bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a41029c0 0x7f77a419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77aa2bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 0x7f77a41a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77aa2bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d2e0 con 0x7f77a4103380 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77aa2bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d160 con 0x7f77a41017d0 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77aa2bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d460 con 0x7f77a41029c0 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 0x7f77a41a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 0x7f77a41a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34862/0 (socket says 172.21.15.31:34862) 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 -- 172.21.15.31:0/3757558311 learned_addr learned my addr 172.21.15.31:0/3757558311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a37fe640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a41029c0 0x7f77a419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a3fff640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41017d0 0x7f77a4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 -- 172.21.15.31:0/3757558311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a41029c0 msgr2=0x7f77a419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a41029c0 0x7f77a419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 -- 172.21.15.31:0/3757558311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41017d0 msgr2=0x7f77a4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41017d0 0x7f77a4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 -- 172.21.15.31:0/3757558311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a41a1710 con 0x7f77a4103380 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.223+0000 7f77a885a640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 0x7f77a41a10a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f779c00eaa0 tx=0x7f779c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.224+0000 7f77a17fa640 1 -- 172.21.15.31:0/3757558311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c00cdb0 con 0x7f77a4103380 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.224+0000 7f77a17fa640 1 -- 172.21.15.31:0/3757558311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779c005020 con 0x7f77a4103380 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.224+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a1a00 con 0x7f77a4103380 2024-04-25T00:32:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.224+0000 7f77a17fa640 1 -- 172.21.15.31:0/3757558311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0116d0 con 0x7f77a4103380 2024-04-25T00:32:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.224+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a1ed0 con 0x7f77a4103380 2024-04-25T00:32:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77a17fa640 1 -- 172.21.15.31:0/3757558311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f779c0051d0 con 0x7f77a4103380 2024-04-25T00:32:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 Processor -- start 2024-04-25T00:32:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 -- start start 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568ee50 0x564b8568f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b85694340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 0x564b85696e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564b855f2cb0 con 0x564b856949b0 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b8559b5b0 con 0x564b8568ee50 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.226+0000 7f77acfcf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564b855eec70 con 0x564b8568f840 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 0x564b85696e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 0x564b85696e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34876/0 (socket says 172.21.15.31:34876) 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a37fe640 1 -- 172.21.15.31:0/581193130 learned_addr learned my addr 172.21.15.31:0/581193130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b85694340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a37fe640 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568ee50 0x564b8568f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a17fa640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f77900a2c30 0x7f77900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 -- 172.21.15.31:0/581193130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568ee50 msgr2=0x564b8568f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568ee50 0x564b8568f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 -- 172.21.15.31:0/581193130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 msgr2=0x564b85696e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 0x564b85696e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 -- 172.21.15.31:0/581193130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b85307210 con 0x564b8568f840 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a37fe640 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 0x564b85696e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a885a640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f77900a2c30 0x7f77900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a17fa640 1 -- 172.21.15.31:0/3757558311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f779c0cf270 con 0x7f77a4103380 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f77a3fff640 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b85694340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f778c00bbb0 tx=0x7f778c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff671db99814281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.227+0000 7f779a7fc640 1 -- 172.21.15.31:0/581193130 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c015070 con 0x564b8568f840 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77a885a640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f77900a2c30 0x7f77900a50f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f779c09b7a0 tx=0x7f779c0c6630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f779a7fc640 1 -- 172.21.15.31:0/581193130 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f778c005020 con 0x564b8568f840 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f779a7fc640 1 -- 172.21.15.31:0/581193130 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c004a00 con 0x564b8568f840 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 -- 172.21.15.31:0/581193130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 msgr2=0x564b85694340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b85694340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f778c00bbb0 tx=0x7f778c00f730 comp rx=0 tx=0).stop 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 -- 172.21.15.31:0/581193130 shutdown_connections 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b856949b0 0x564b85696e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b85694340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 --2- 172.21.15.31:0/581193130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568ee50 0x564b8568f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 -- 172.21.15.31:0/581193130 >> 172.21.15.31:0/581193130 conn(0x564b8568a4c0 msgr2=0x564b8568a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 -- 172.21.15.31:0/581193130 shutdown_connections 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.228+0000 7f77acfcf740 1 -- 172.21.15.31:0/581193130 wait complete. 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 -- start start 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 Processor -- start 2024-04-25T00:32:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b856992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b856997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b856a0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564b85697960 con 0x564b8568ee50 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b855aeb50 con 0x564b856949b0 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564b85697ae0 con 0x564b8568f840 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77acfcf740 10 client.0 _ll_register_callbacks cb 0x564b855e9ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b856992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b856992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34880/0 (socket says 172.21.15.31:34880) 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77a37fe640 1 -- 172.21.15.31:0/4116962387 learned_addr learned my addr 172.21.15.31:0/4116962387 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.229+0000 7f77a3fff640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b856997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b856a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 -- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 msgr2=0x564b856997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b856997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 -- 172.21.15.31:0/4116962387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 msgr2=0x564b856a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b856a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 -- 172.21.15.31:0/4116962387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b853d3620 con 0x564b8568ee50 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77a37fe640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b856992a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7794009230 tx=0x7f779402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb07dec41747bf57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77837fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77837fe640 1 -- 172.21.15.31:0/4116962387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794035040 con 0x564b8568ee50 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77837fe640 1 -- 172.21.15.31:0/4116962387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f77940035f0 con 0x564b8568ee50 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778c005020 con 0x564b8568ee50 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77837fe640 1 -- 172.21.15.31:0/4116962387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77940037f0 con 0x564b8568ee50 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77acfcf740 10 client.15531 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b856a3c00 con 0x564b8568ee50 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.230+0000 7f77837fe640 1 -- 172.21.15.31:0/4116962387 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f778c005020 con 0x564b8568ee50 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77837fe640 1 -- 172.21.15.31:0/4116962387 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f779403d8a0 con 0x564b8568ee50 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77acfcf740 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77acfcf740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77acfcf740 20 client.15531 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77acfcf740 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b856a52d0 0x564b856a7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77acfcf740 4 client.15531 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b856001f0 con 0x564b856a52d0 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77a3fff640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b856a52d0 0x564b856a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77837fe640 10 client.15531 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.231+0000 7f77a3fff640 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b856a52d0 0x564b856a7720 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b69e6f71dde316 server_cookie=eefe39a92d01ed22 in_seq=0 out_seq=0 2024-04-25T00:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.232+0000 7f77837fe640 1 -- 172.21.15.31:0/4116962387 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b856001f0 con 0x564b856a52d0 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.232+0000 7f77837fe640 10 client.15531 handle_command_reply: tid=0 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.232+0000 7f77acfcf740 1 client.15531 shutdown 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 msgr2=0x564b856992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b856992a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7794009230 tx=0x7f779402a410 comp rx=0 tx=0).stop 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 shutdown_connections 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b856a52d0 0x564b856a7720 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b856a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568f840 0x564b856997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77837fe640 0 client.15531 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 --2- 172.21.15.31:0/4116962387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b856992a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 >> 172.21.15.31:0/4116962387 conn(0x564b8568a4c0 msgr2=0x564b856895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 shutdown_connections 2024-04-25T00:32:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 1 -- 172.21.15.31:0/4116962387 wait complete. 2024-04-25T00:32:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.233+0000 7f77acfcf740 20 client.15531 trim_cache size 0 max 16384 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 Processor -- start 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 -- start start 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b85694270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568f840 0x564b856a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b856949b0 0x564b8568db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41aad70 con 0x564b8568ee50 2024-04-25T00:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41ac8b0 con 0x564b8568f840 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77acfcf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41ab440 con 0x564b856949b0 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b85694270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b85694270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34896/0 (socket says 172.21.15.31:34896) 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 -- 172.21.15.31:0/3349834498 learned_addr learned my addr 172.21.15.31:0/3349834498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 -- 172.21.15.31:0/3349834498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b856949b0 msgr2=0x564b8568db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b856949b0 0x564b8568db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 -- 172.21.15.31:0/3349834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568f840 msgr2=0x564b856a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a3fff640 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568f840 0x564b856a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568f840 0x564b856a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.248+0000 7f77a37fe640 1 -- 172.21.15.31:0/3349834498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b85693ef0 con 0x564b8568ee50 2024-04-25T00:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77a37fe640 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b85694270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f77940092a0 tx=0x7f7794009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf68bcb7ece4c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f7799ffb640 1 -- 172.21.15.31:0/3349834498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794035070 con 0x564b8568ee50 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f7799ffb640 1 -- 172.21.15.31:0/3349834498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779404b070 con 0x564b8568ee50 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f7799ffb640 1 -- 172.21.15.31:0/3349834498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779403d6c0 con 0x564b8568ee50 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 -- 172.21.15.31:0/3349834498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 msgr2=0x564b85694270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b85694270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f77940092a0 tx=0x7f7794009830 comp rx=0 tx=0).stop 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 -- 172.21.15.31:0/3349834498 shutdown_connections 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b856949b0 0x564b8568db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b8568f840 0x564b856a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 --2- 172.21.15.31:0/3349834498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568ee50 0x564b85694270 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.249+0000 7f77acfcf740 1 -- 172.21.15.31:0/3349834498 >> 172.21.15.31:0/3349834498 conn(0x564b8568a4c0 msgr2=0x564b8568d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 -- 172.21.15.31:0/3349834498 shutdown_connections 2024-04-25T00:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 -- 172.21.15.31:0/3349834498 wait complete. 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 -- start start 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 Processor -- start 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568ee50 0x564b8569dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 0x564b8569e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b8569f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41ac1e0 con 0x564b8568f840 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41abb10 con 0x564b856949b0 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41a8d50 con 0x564b8568ee50 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77acfcf740 10 client.0 _ll_register_callbacks cb 0x7f7794031770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.250+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 0x564b8569e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 0x564b8569e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34904/0 (socket says 172.21.15.31:34904) 2024-04-25T00:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 -- 172.21.15.31:0/250149241 learned_addr learned my addr 172.21.15.31:0/250149241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a37fe640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568ee50 0x564b8569dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a37fe640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b8569f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 -- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568ee50 msgr2=0x564b8569dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568ee50 0x564b8569dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 -- 172.21.15.31:0/250149241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 msgr2=0x564b8569f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b8569f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 -- 172.21.15.31:0/250149241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8569f710 con 0x564b8568f840 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77a3fff640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 0x564b8569e110 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f778c00b970 tx=0x7f778c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa179cc9403a73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77837fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77837fe640 1 -- 172.21.15.31:0/250149241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c015040 con 0x564b8568f840 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77837fe640 1 -- 172.21.15.31:0/250149241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f778c027020 con 0x564b8568f840 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779404b070 con 0x564b8568f840 2024-04-25T00:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77837fe640 1 -- 172.21.15.31:0/250149241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f778c00be70 con 0x564b8568f840 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77acfcf740 10 client.15537 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.251+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b856a0ac0 con 0x564b8568f840 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77837fe640 1 -- 172.21.15.31:0/250149241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779404b070 con 0x564b8568f840 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77837fe640 1 -- 172.21.15.31:0/250149241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f778c004710 con 0x564b8568f840 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77acfcf740 10 client.15537 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77acfcf740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77acfcf740 20 client.15537 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77acfcf740 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b8570bda0 0x564b8570e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77acfcf740 4 client.15537 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b855fa540 con 0x564b8570bda0 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77a37fe640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b8570bda0 0x564b8570e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77a37fe640 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b8570bda0 0x564b8570e1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e538f1efe6473afd server_cookie=4493405dac81c077 in_seq=0 out_seq=0 2024-04-25T00:32:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.252+0000 7f77837fe640 10 client.15537 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77837fe640 1 -- 172.21.15.31:0/250149241 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b855fa540 con 0x564b8570bda0 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77837fe640 10 client.15537 handle_command_reply: tid=0 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 client.15537 shutdown 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 msgr2=0x564b8569e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 0x564b8569e110 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f778c00b970 tx=0x7f778c00bb40 comp rx=0 tx=0).stop 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 shutdown_connections 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564b8570bda0 0x564b8570e1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b856949b0 0x564b8569f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564b8568f840 0x564b8569e110 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77837fe640 0 client.15537 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 --2- 172.21.15.31:0/250149241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b8568ee50 0x564b8569dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.266+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 >> 172.21.15.31:0/250149241 conn(0x564b8568a4c0 msgr2=0x564b85688d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 shutdown_connections 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77acfcf740 1 -- 172.21.15.31:0/250149241 wait complete. 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77acfcf740 20 client.15537 trim_cache size 0 max 16384 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:32:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:32:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f77900a2c30 msgr2=0x7f77900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f77900a2c30 0x7f77900a50f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f779c09b7a0 tx=0x7f779c0c6630 comp rx=0 tx=0).stop 2024-04-25T00:32:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 msgr2=0x7f77a41a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 0x7f77a41a10a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f779c00eaa0 tx=0x7f779c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 shutdown_connections 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f77900a2c30 0x7f77900a50f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77a4103380 0x7f77a41a10a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a41029c0 0x7f77a419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 --2- 172.21.15.31:0/3757558311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41017d0 0x7f77a4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 >> 172.21.15.31:0/3757558311 conn(0x7f77a40fcd60 msgr2=0x7f77a4073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 shutdown_connections 2024-04-25T00:32:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:28.267+0000 7f77aa2bd640 1 -- 172.21.15.31:0/3757558311 wait complete. 2024-04-25T00:32:28.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:28 smithi031 ceph-mon[29167]: pgmap v418: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:28 smithi169 ceph-mon[33112]: pgmap v418: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:28 smithi040 ceph-mon[32945]: pgmap v418: 97 pgs: 97 active+clean; 1.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:30 smithi031 ceph-mon[29167]: pgmap v419: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:32:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:30 smithi169 ceph-mon[33112]: pgmap v419: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:32:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:30 smithi040 ceph-mon[32945]: pgmap v419: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:32:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:32 smithi031 ceph-mon[29167]: pgmap v420: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:32:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:32 smithi169 ceph-mon[33112]: pgmap v420: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:32:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:32 smithi040 ceph-mon[32945]: pgmap v420: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:32:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:34 smithi031 ceph-mon[29167]: pgmap v421: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:32:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:34 smithi169 ceph-mon[33112]: pgmap v421: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:32:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:34 smithi040 ceph-mon[32945]: pgmap v421: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:32:36.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:36 smithi031 ceph-mon[29167]: pgmap v422: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:36 smithi169 ceph-mon[33112]: pgmap v422: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:36 smithi040 ceph-mon[32945]: pgmap v422: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:38 smithi031 ceph-mon[29167]: pgmap v423: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:38 smithi169 ceph-mon[33112]: pgmap v423: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:38 smithi040 ceph-mon[32945]: pgmap v423: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:40 smithi031 ceph-mon[29167]: pgmap v424: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:32:40.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:40 smithi169 ceph-mon[33112]: pgmap v424: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:32:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:40 smithi040 ceph-mon[32945]: pgmap v424: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:32:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 -- 172.21.15.31:0/2248172137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c0ffe50 msgr2=0x7f4a3c100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 --2- 172.21.15.31:0/2248172137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c0ffe50 0x7f4a3c100270 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009980 tx=0x7f4a2c02c620 comp rx=0 tx=0).stop 2024-04-25T00:32:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 -- 172.21.15.31:0/2248172137 shutdown_connections 2024-04-25T00:32:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 --2- 172.21.15.31:0/2248172137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3c101960 0x7f4a3c108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 --2- 172.21.15.31:0/2248172137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 --2- 172.21.15.31:0/2248172137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c0ffe50 0x7f4a3c100270 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 -- 172.21.15.31:0/2248172137 >> 172.21.15.31:0/2248172137 conn(0x7f4a3c0fb3c0 msgr2=0x7f4a3c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 -- 172.21.15.31:0/2248172137 shutdown_connections 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.367+0000 7f4a41d29640 1 -- 172.21.15.31:0/2248172137 wait complete. 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 Processor -- start 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 -- start start 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3c0ffe50 0x7f4a3c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c101960 0x7f4a3c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c10b130 con 0x7f4a3c100fa0 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c10afb0 con 0x7f4a3c0ffe50 2024-04-25T00:32:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a41d29640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c10b2b0 con 0x7f4a3c101960 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35190/0 (socket says 172.21.15.31:35190) 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.368+0000 7f4a33fff640 1 -- 172.21.15.31:0/3470597371 learned_addr learned my addr 172.21.15.31:0/3470597371 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3c0ffe50 0x7f4a3c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c101960 0x7f4a3c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a33fff640 1 -- 172.21.15.31:0/3470597371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c101960 msgr2=0x7f4a3c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a33fff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c101960 0x7f4a3c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a33fff640 1 -- 172.21.15.31:0/3470597371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3c0ffe50 msgr2=0x7f4a3c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a33fff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3c0ffe50 0x7f4a3c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a33fff640 1 -- 172.21.15.31:0/3470597371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c1a1700 con 0x7f4a3c100fa0 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c101960 0x7f4a3c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a33fff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c19a020 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400db70 tx=0x7f4a2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a397fa640 1 -- 172.21.15.31:0/3470597371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2401b070 con 0x7f4a3c100fa0 2024-04-25T00:32:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a397fa640 1 -- 172.21.15.31:0/3470597371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2400ae50 con 0x7f4a3c100fa0 2024-04-25T00:32:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c1a19f0 con 0x7f4a3c100fa0 2024-04-25T00:32:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a397fa640 1 -- 172.21.15.31:0/3470597371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24016960 con 0x7f4a3c100fa0 2024-04-25T00:32:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.369+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1a1ef0 con 0x7f4a3c100fa0 2024-04-25T00:32:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.372+0000 7f4a397fa640 1 -- 172.21.15.31:0/3470597371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4a2401e070 con 0x7f4a3c100fa0 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.372+0000 7f4a44a3c740 1 Processor -- start 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.372+0000 7f4a44a3c740 1 -- start start 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c91340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c919b0 0x564484c93e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564484befcb0 con 0x564484c8be50 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564484b985b0 con 0x564484c8c840 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564484bebc70 con 0x564484c919b0 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35192/0 (socket says 172.21.15.31:35192) 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 -- 172.21.15.31:0/2457920003 learned_addr learned my addr 172.21.15.31:0/2457920003 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a397fa640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a280a2c30 0x7f4a280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c91340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a33fff640 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c919b0 0x564484c93e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a280a2c30 0x7f4a280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 -- 172.21.15.31:0/2457920003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c919b0 msgr2=0x564484c93e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a397fa640 1 -- 172.21.15.31:0/3470597371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a240cef80 con 0x7f4a3c100fa0 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c919b0 0x564484c93e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 -- 172.21.15.31:0/2457920003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 msgr2=0x564484c91340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c91340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3bfff640 1 -- 172.21.15.31:0/2457920003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564484904210 con 0x564484c8be50 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.373+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c91340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.374+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8c270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a340129e0 tx=0x7f4a34012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da824ab3d5fd327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.374+0000 7f4a31ffb640 1 -- 172.21.15.31:0/2457920003 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34004d40 con 0x564484c8be50 2024-04-25T00:32:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.374+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a280a2c30 0x7f4a280a50f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a34020000 tx=0x7f4a3401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.374+0000 7f4a31ffb640 1 -- 172.21.15.31:0/2457920003 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a34021070 con 0x564484c8be50 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.374+0000 7f4a31ffb640 1 -- 172.21.15.31:0/2457920003 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400fa10 con 0x564484c8be50 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 -- 172.21.15.31:0/2457920003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 msgr2=0x564484c8c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8c270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a340129e0 tx=0x7f4a34012ea0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 -- 172.21.15.31:0/2457920003 shutdown_connections 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c919b0 0x564484c93e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c91340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/2457920003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8c270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 -- 172.21.15.31:0/2457920003 >> 172.21.15.31:0/2457920003 conn(0x564484c874c0 msgr2=0x564484c878c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 -- 172.21.15.31:0/2457920003 shutdown_connections 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 -- 172.21.15.31:0/2457920003 wait complete. 2024-04-25T00:32:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.375+0000 7f4a44a3c740 1 -- start start 2024-04-25T00:32:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 Processor -- start 2024-04-25T00:32:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8be50 0x564484c96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c96780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484c9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564484c94960 con 0x564484c919b0 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564484babb50 con 0x564484c8be50 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564484c94ae0 con 0x564484c8c840 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a44a3c740 10 client.0 _ll_register_callbacks cb 0x564484be6ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484c9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484c9d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35208/0 (socket says 172.21.15.31:35208) 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 -- 172.21.15.31:0/169331153 learned_addr learned my addr 172.21.15.31:0/169331153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a33fff640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c96780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8be50 0x564484c96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 -- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 msgr2=0x564484c96780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c96780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 -- 172.21.15.31:0/169331153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8be50 msgr2=0x564484c96240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8be50 0x564484c96240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 -- 172.21.15.31:0/169331153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644849d0620 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484c9d7c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400fa00 tx=0x7f4a3400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c32da888c9cdda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.376+0000 7f4a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a1affd640 1 -- 172.21.15.31:0/169331153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34034060 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a1affd640 1 -- 172.21.15.31:0/169331153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a34004c10 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a34021070 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a1affd640 1 -- 172.21.15.31:0/169331153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a34004e10 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564484ca0ba0 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a1affd640 1 -- 172.21.15.31:0/169331153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a34021070 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a1affd640 1 -- 172.21.15.31:0/169331153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a340105f0 con 0x564484c919b0 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 20 client.15549 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484ca2000 0x564484ca43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 4 client.15549 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.377+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564484bfd1f0 con 0x564484ca2000 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.378+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484ca2000 0x564484ca43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.378+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484ca2000 0x564484ca43f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55330b59bf93c14 server_cookie=ea9b51d9cbcbb827 in_seq=0 out_seq=0 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.378+0000 7f4a1affd640 10 client.15549 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a1affd640 1 -- 172.21.15.31:0/169331153 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564484bfd1f0 con 0x564484ca2000 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a1affd640 10 client.15549 handle_command_reply: tid=0 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 client.15549 shutdown 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 msgr2=0x564484c9d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484c9d7c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400fa00 tx=0x7f4a3400fa30 comp rx=0 tx=0).stop 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 shutdown_connections 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484ca2000 0x564484ca43f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484c9d7c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c96780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a1affd640 0 client.15549 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/169331153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8be50 0x564484c96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 >> 172.21.15.31:0/169331153 conn(0x564484c874c0 msgr2=0x564484c865c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 shutdown_connections 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 1 -- 172.21.15.31:0/169331153 wait complete. 2024-04-25T00:32:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.379+0000 7f4a44a3c740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 Processor -- start 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 -- start start 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c8e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c919b0 0x564484c9d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1aacd0 con 0x564484c8be50 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1ac810 con 0x564484c919b0 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.394+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1ab3a0 con 0x564484c8c840 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35212/0 (socket says 172.21.15.31:35212) 2024-04-25T00:32:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 -- 172.21.15.31:0/1282804253 learned_addr learned my addr 172.21.15.31:0/1282804253 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c919b0 0x564484c9d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a33fff640 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c8e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 -- 172.21.15.31:0/1282804253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 msgr2=0x564484c8e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c8e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 -- 172.21.15.31:0/1282804253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c919b0 msgr2=0x564484c9d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c919b0 0x564484c9d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 -- 172.21.15.31:0/1282804253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564484c90ef0 con 0x564484c8be50 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8dc00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c02cb40 tx=0x7f4a2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5aa8c7c294e790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a317fa640 1 -- 172.21.15.31:0/1282804253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c04a020 con 0x564484c8be50 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a317fa640 1 -- 172.21.15.31:0/1282804253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c0346c0 con 0x564484c8be50 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.395+0000 7f4a317fa640 1 -- 172.21.15.31:0/1282804253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c037040 con 0x564484c8be50 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 -- 172.21.15.31:0/1282804253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 msgr2=0x564484c8dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8dc00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c02cb40 tx=0x7f4a2c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 -- 172.21.15.31:0/1282804253 shutdown_connections 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c919b0 0x564484c9d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8c840 0x564484c8e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/1282804253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c8be50 0x564484c8dc00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 -- 172.21.15.31:0/1282804253 >> 172.21.15.31:0/1282804253 conn(0x564484c874c0 msgr2=0x564484c878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 -- 172.21.15.31:0/1282804253 shutdown_connections 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 -- 172.21.15.31:0/1282804253 wait complete. 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.396+0000 7f4a44a3c740 1 -- start start 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 Processor -- start 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8be50 0x564484c9a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c9ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484d06510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1ac140 con 0x564484c919b0 2024-04-25T00:32:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1aba70 con 0x564484c8c840 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1a8cb0 con 0x564484c8be50 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a44a3c740 10 client.0 _ll_register_callbacks cb 0x7f4a2c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484d06510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484d06510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35226/0 (socket says 172.21.15.31:35226) 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 -- 172.21.15.31:0/3042835960 learned_addr learned my addr 172.21.15.31:0/3042835960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a33fff640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c9ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8be50 0x564484c9a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 -- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8be50 msgr2=0x564484c9a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8be50 0x564484c9a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 -- 172.21.15.31:0/3042835960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 msgr2=0x564484c9ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c9ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 -- 172.21.15.31:0/3042835960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564484d06a50 con 0x564484c919b0 2024-04-25T00:32:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8be50 0x564484c9a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a3bfff640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484d06510 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a34010040 tx=0x7f4a34012fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9a871bf7cf1325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.397+0000 7f4a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a1affd640 1 -- 172.21.15.31:0/3042835960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34034070 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a1affd640 1 -- 172.21.15.31:0/3042835960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a34004c10 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c0346c0 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a1affd640 1 -- 172.21.15.31:0/3042835960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a3404a050 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 10 client.15555 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564484d07c70 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a1affd640 1 -- 172.21.15.31:0/3042835960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c0346c0 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a1affd640 1 -- 172.21.15.31:0/3042835960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a34041070 con 0x564484c919b0 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 10 client.15555 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 20 client.15555 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484d085f0 0x564484d0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 4 client.15555 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564484bf7540 con 0x564484d085f0 2024-04-25T00:32:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.398+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484d085f0 0x564484d0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.399+0000 7f4a3b7fe640 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484d085f0 0x564484d0aa10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15763314b04edabd server_cookie=10585e4c71ccba08 in_seq=0 out_seq=0 2024-04-25T00:32:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.399+0000 7f4a1affd640 10 client.15555 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.413+0000 7f4a1affd640 1 -- 172.21.15.31:0/3042835960 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564484bf7540 con 0x564484d085f0 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.413+0000 7f4a1affd640 10 client.15555 handle_command_reply: tid=0 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.413+0000 7f4a44a3c740 1 client.15555 shutdown 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.413+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 msgr2=0x564484d06510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.413+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484d06510 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a34010040 tx=0x7f4a34012fd0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 shutdown_connections 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564484d085f0 0x564484d0aa10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564484c919b0 0x564484d06510 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564484c8c840 0x564484c9ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 --2- 172.21.15.31:0/3042835960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564484c8be50 0x564484c9a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a1affd640 0 client.15555 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 >> 172.21.15.31:0/3042835960 conn(0x564484c874c0 msgr2=0x564484c8dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 shutdown_connections 2024-04-25T00:32:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 1 -- 172.21.15.31:0/3042835960 wait complete. 2024-04-25T00:32:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a44a3c740 20 client.15555 trim_cache size 0 max 16384 2024-04-25T00:32:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:32:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:32:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:32:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a280a2c30 msgr2=0x7f4a280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a280a2c30 0x7f4a280a50f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a34020000 tx=0x7f4a3401e000 comp rx=0 tx=0).stop 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 msgr2=0x7f4a3c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c19a020 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400db70 tx=0x7f4a2400a620 comp rx=0 tx=0).stop 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 shutdown_connections 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a3c101960 0x7f4a3c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a280a2c30 0x7f4a280a50f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a3c100fa0 0x7f4a3c19a020 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 --2- 172.21.15.31:0/3470597371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3c0ffe50 0x7f4a3c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 >> 172.21.15.31:0/3470597371 conn(0x7f4a3c0fb3c0 msgr2=0x7f4a3c190810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.414+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 shutdown_connections 2024-04-25T00:32:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:41.415+0000 7f4a41d29640 1 -- 172.21.15.31:0/3470597371 wait complete. 2024-04-25T00:32:42.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:42 smithi031 ceph-mon[29167]: pgmap v425: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:42 smithi169 ceph-mon[33112]: pgmap v425: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:42 smithi040 ceph-mon[32945]: pgmap v425: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:44.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:44 smithi031 ceph-mon[29167]: pgmap v426: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:32:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:44 smithi169 ceph-mon[33112]: pgmap v426: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:32:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:44 smithi040 ceph-mon[32945]: pgmap v426: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:32:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:46 smithi031 ceph-mon[29167]: pgmap v427: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:46 smithi169 ceph-mon[33112]: pgmap v427: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:46 smithi040 ceph-mon[32945]: pgmap v427: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:32:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:48 smithi031 ceph-mon[29167]: pgmap v428: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:32:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:32:48.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:48 smithi169 ceph-mon[33112]: pgmap v428: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:32:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:48.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:32:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:48 smithi040 ceph-mon[32945]: pgmap v428: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:32:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:32:50.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:50 smithi031 ceph-mon[29167]: pgmap v429: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:50.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:50 smithi169 ceph-mon[33112]: pgmap v429: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:50 smithi040 ceph-mon[32945]: pgmap v429: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:51.607 INFO:tasks.workunit.client.0.smithi031.stdout:1/600: open fd = 3 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:2/600: open fd = 3 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:3/600: open fd = 3 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:4/600: open fd = 3 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:5/600: open fd = 3 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:6/600: open fd = 3 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:7/600: open fd = 3 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:8/600: open fd = 3 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:9/600: open fd = 3 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:10/600: open fd = 3 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:11/600: open fd = 3 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:12/600: open fd = 3 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:13/600: open fd = 3 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:14/600: open fd = 3 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:15/600: open fd = 3 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:16/600: open fd = 3 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:17/600: open fd = 3 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:18/600: open fd = 3 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:19/600: open fd = 3 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:20/600: open fd = 3 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:21/600: open fd = 3 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:22/600: open fd = 3 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:23/600: open fd = 3 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:24/600: open fd = 3 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.613 INFO:tasks.workunit.client.0.smithi031.stdout:25/600: open fd = 3 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:26/600: open fd = 3 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:27/600: open fd = 3 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:28/600: open fd = 3 2024-04-25T00:32:51.614 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:29/600: open fd = 3 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:30/600: open fd = 3 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:31/600: open fd = 3 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:32/600: open fd = 3 2024-04-25T00:32:51.615 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:33/600: open fd = 3 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:34/600: open fd = 3 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:35/600: open fd = 3 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:36/600: open fd = 3 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.616 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:37/600: open fd = 3 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:38/600: open fd = 3 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:39/600: open fd = 3 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:40/600: open fd = 3 2024-04-25T00:32:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:41/600: open fd = 3 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:42/600: open fd = 3 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:43/600: open fd = 3 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:44/600: open fd = 3 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:45/600: open fd = 3 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:46/600: open fd = 3 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:47/600: open fd = 3 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:48/600: open fd = 3 2024-04-25T00:32:51.619 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:49/600: open fd = 3 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:50/600: open fd = 3 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:51/600: open fd = 3 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.620 INFO:tasks.workunit.client.0.smithi031.stdout:52/600: open fd = 3 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:53/600: open fd = 3 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:54/600: open fd = 3 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:55/600: open fd = 3 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:56/600: open fd = 3 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:57/600: open fd = 3 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:58/600: open fd = 3 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:59/600: open fd = 3 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:60/600: open fd = 3 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:61/600: open fd = 3 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:62/600: open fd = 3 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.623 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.624 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:51.624 INFO:tasks.workunit.client.0.smithi031.stdout:63/600: open fd = 3 2024-04-25T00:32:51.624 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.624 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:32:51.624 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:32:52.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:52 smithi031 ceph-mon[29167]: pgmap v430: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:52.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:52 smithi169 ceph-mon[33112]: pgmap v430: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:52.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:52 smithi040 ceph-mon[32945]: pgmap v430: 97 pgs: 97 active+clean; 1.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/572807665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b4101890 msgr2=0x7ff9b4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/572807665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b4101890 0x7ff9b4101d10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009900 tx=0x7ff9a802a410 comp rx=0 tx=0).stop 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/572807665 shutdown_connections 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/572807665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9b4102250 0x7ff9b4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/572807665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b4101890 0x7ff9b4101d10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/572807665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9b41006a0 0x7ff9b4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/572807665 >> 172.21.15.31:0/572807665 conn(0x7ff9b40fbc10 msgr2=0x7ff9b40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/572807665 shutdown_connections 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/572807665 wait complete. 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.514+0000 7ff9b9ea5640 1 Processor -- start 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 -- start start 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 0x7ff9b410b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9b4101890 0x7ff9b4109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9b4102250 0x7ff9b4109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410d2f0 con 0x7ff9b41006a0 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410d170 con 0x7ff9b4101890 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b9ea5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410d470 con 0x7ff9b4102250 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 0x7ff9b410b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 0x7ff9b410b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36612/0 (socket says 172.21.15.31:36612) 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 -- 172.21.15.31:0/1480980205 learned_addr learned my addr 172.21.15.31:0/1480980205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9b4102250 0x7ff9b4109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 -- 172.21.15.31:0/1480980205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9b4102250 msgr2=0x7ff9b4109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9b4102250 0x7ff9b4109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 -- 172.21.15.31:0/1480980205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9b4101890 msgr2=0x7ff9b4109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9b4101890 0x7ff9b4109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9b4101890 0x7ff9b4109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 -- 172.21.15.31:0/1480980205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b410a430 con 0x7ff9b41006a0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.515+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 0x7ff9b410b020 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400d950 tx=0x7ff9a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.516+0000 7ff9b0ff9640 1 -- 172.21.15.31:0/1480980205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4017020 con 0x7ff9b41006a0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.516+0000 7ff9b0ff9640 1 -- 172.21.15.31:0/1480980205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4009c30 con 0x7ff9b41006a0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.516+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b41a1a20 con 0x7ff9b41006a0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.516+0000 7ff9b0ff9640 1 -- 172.21.15.31:0/1480980205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400f6b0 con 0x7ff9b41006a0 2024-04-25T00:32:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.516+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b41a1fc0 con 0x7ff9b41006a0 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b0ff9640 1 -- 172.21.15.31:0/1480980205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff9a401a020 con 0x7ff9b41006a0 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 Processor -- start 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 -- start start 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a48e50 0x55f9b0a49270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 0x55f9b0a4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a50e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9b09accb0 con 0x55f9b0a49840 2024-04-25T00:32:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f9b09555b0 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f9b09a8c70 con 0x55f9b0a48e50 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 0x55f9b0a4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 0x55f9b0a4e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36614/0 (socket says 172.21.15.31:36614) 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3033458520 learned_addr learned my addr 172.21.15.31:0/3033458520 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a50e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a48e50 0x55f9b0a49270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.519+0000 7ff9b0ff9640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff99c0a2cb0 0x7ff99c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff99c0a2cb0 0x7ff99c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3033458520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a48e50 msgr2=0x55f9b0a49270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a48e50 0x55f9b0a49270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3033458520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 msgr2=0x55f9b0a50e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a50e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3033458520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b06c1210 con 0x55f9b0a49840 2024-04-25T00:32:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b0ff9640 1 -- 172.21.15.31:0/1480980205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff9a40cedb0 con 0x7ff9b41006a0 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 0x55f9b0a4e340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4097fb0 tx=0x7ff9a40979b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a0e76e58f2e3cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff99c0a2cb0 0x7ff99c0a5170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009fd0 tx=0x7ff9ac004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff999ffb640 1 -- 172.21.15.31:0/3033458520 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4009de0 con 0x55f9b0a49840 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff999ffb640 1 -- 172.21.15.31:0/3033458520 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a409bce0 con 0x55f9b0a49840 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.520+0000 7ff999ffb640 1 -- 172.21.15.31:0/3033458520 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4019e30 con 0x55f9b0a49840 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3033458520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 msgr2=0x55f9b0a4e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 0x55f9b0a4e340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4097fb0 tx=0x7ff9a40979b0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3033458520 shutdown_connections 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a49840 0x55f9b0a4e340 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3033458520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a48e50 0x55f9b0a49270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3033458520 >> 172.21.15.31:0/3033458520 conn(0x55f9b0a444c0 msgr2=0x55f9b0a448c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3033458520 shutdown_connections 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3033458520 wait complete. 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.521+0000 7ff9bcbb9740 1 -- start start 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 Processor -- start 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a49840 0x55f9b0a53560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a5a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9b0a51780 con 0x55f9b0a48e50 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f9b0968b50 con 0x55f9b0a49840 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f9b0a51900 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9bcbb9740 10 client.0 _ll_register_callbacks cb 0x55f9b09a3ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a53020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a53020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36618/0 (socket says 172.21.15.31:36618) 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3943288326 learned_addr learned my addr 172.21.15.31:0/3943288326 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a49840 0x55f9b0a53560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a5a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a4e9b0 msgr2=0x55f9b0a5a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a5a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3943288326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a49840 msgr2=0x55f9b0a53560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a49840 0x55f9b0a53560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b37fe640 1 -- 172.21.15.31:0/3943288326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b078d620 con 0x55f9b0a48e50 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.522+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a5a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a53020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40cc8d0 tx=0x7ff9a409b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d6015f609534c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff992ffd640 1 -- 172.21.15.31:0/3943288326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4009de0 con 0x55f9b0a48e50 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff992ffd640 1 -- 172.21.15.31:0/3943288326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff9a4097c50 con 0x55f9b0a48e50 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a409bce0 con 0x55f9b0a48e50 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff992ffd640 1 -- 172.21.15.31:0/3943288326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9a4019e10 con 0x55f9b0a48e50 2024-04-25T00:32:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 10 client.15567 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9b0a51780 con 0x55f9b0a48e50 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff992ffd640 1 -- 172.21.15.31:0/3943288326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a409bce0 con 0x55f9b0a48e50 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff992ffd640 1 -- 172.21.15.31:0/3943288326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9a4024060 con 0x55f9b0a48e50 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 10 client.15567 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 20 client.15567 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0a5ed60 0x55f9b0a61150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 4 client.15567 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.523+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b09ba1f0 con 0x55f9b0a5ed60 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.524+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0a5ed60 0x55f9b0a61150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.524+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0a5ed60 0x55f9b0a61150 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc14888c5fd22c3a server_cookie=4fa86665b314e225 in_seq=0 out_seq=0 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.524+0000 7ff992ffd640 10 client.15567 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff992ffd640 1 -- 172.21.15.31:0/3943288326 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9b09ba1f0 con 0x55f9b0a5ed60 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff992ffd640 10 client.15567 handle_command_reply: tid=0 2024-04-25T00:32:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 client.15567 shutdown 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 msgr2=0x55f9b0a53020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a53020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40cc8d0 tx=0x7ff9a409b030 comp rx=0 tx=0).stop 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 shutdown_connections 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0a5ed60 0x55f9b0a61150 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a5a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a49840 0x55f9b0a53560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3943288326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a53020 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff992ffd640 0 client.15567 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 >> 172.21.15.31:0/3943288326 conn(0x55f9b0a444c0 msgr2=0x55f9b0a435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 shutdown_connections 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3943288326 wait complete. 2024-04-25T00:32:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.525+0000 7ff9bcbb9740 20 client.15567 trim_cache size 0 max 16384 2024-04-25T00:32:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.540+0000 7ff9bcbb9740 1 Processor -- start 2024-04-25T00:32:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.540+0000 7ff9bcbb9740 1 -- start start 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a5a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a52410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41aae50 con 0x55f9b0a48e50 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ac990 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ab520 con 0x55f9b0a49840 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36624/0 (socket says 172.21.15.31:36624) 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 -- 172.21.15.31:0/2116255094 learned_addr learned my addr 172.21.15.31:0/2116255094 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a5a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a52410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 -- 172.21.15.31:0/2116255094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 msgr2=0x55f9b0a5a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a5a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 -- 172.21.15.31:0/2116255094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 msgr2=0x55f9b0a52410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a52410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 -- 172.21.15.31:0/2116255094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b0a61730 con 0x55f9b0a48e50 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a534c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff9a409b2e0 tx=0x7ff9a4019da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec8a3b131c39773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9997fa640 1 -- 172.21.15.31:0/2116255094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a409bce0 con 0x55f9b0a48e50 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9997fa640 1 -- 172.21.15.31:0/2116255094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4097c50 con 0x55f9b0a48e50 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.541+0000 7ff9997fa640 1 -- 172.21.15.31:0/2116255094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40d25c0 con 0x55f9b0a48e50 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/2116255094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 msgr2=0x55f9b0a534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a534c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff9a409b2e0 tx=0x7ff9a4019da0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/2116255094 shutdown_connections 2024-04-25T00:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a52410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a5a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/2116255094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a48e50 0x55f9b0a534c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/2116255094 >> 172.21.15.31:0/2116255094 conn(0x55f9b0a444c0 msgr2=0x55f9b0a51d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/2116255094 shutdown_connections 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/2116255094 wait complete. 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.542+0000 7ff9bcbb9740 1 -- start start 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 Processor -- start 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a48e50 0x55f9b0a578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a57e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a58e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ac2c0 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41abbf0 con 0x55f9b0a48e50 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41a8e30 con 0x55f9b0a49840 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9bcbb9740 10 client.0 _ll_register_callbacks cb 0x7ff9a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a58e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a58e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36636/0 (socket says 172.21.15.31:36636) 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 -- 172.21.15.31:0/3896975217 learned_addr learned my addr 172.21.15.31:0/3896975217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a48e50 0x55f9b0a578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b2ffd640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a57e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 -- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 msgr2=0x55f9b0a57e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a57e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 -- 172.21.15.31:0/3896975217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a48e50 msgr2=0x55f9b0a578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a48e50 0x55f9b0a578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.543+0000 7ff9b3fff640 1 -- 172.21.15.31:0/3896975217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b0a59430 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9b3fff640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a58e40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0117a0 tx=0x7ff9ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d5f704e1446454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:32:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff992ffd640 1 -- 172.21.15.31:0/3896975217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac004d20 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff992ffd640 1 -- 172.21.15.31:0/3896975217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff9ac01e990 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4097c50 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff992ffd640 1 -- 172.21.15.31:0/3896975217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9ac02d030 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 10 client.15573 fetch_fsmap learned FSMap version 26 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9b0a5a810 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff992ffd640 1 -- 172.21.15.31:0/3896975217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4097c50 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff992ffd640 1 -- 172.21.15.31:0/3896975217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9ac01b040 con 0x55f9b0a4e9b0 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 10 client.15573 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 20 client.15573 populate_metadata read hostname 'smithi031' 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0ac5970 0x55f9b0ac7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 4 client.15573 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.544+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9b09b4540 con 0x55f9b0ac5970 2024-04-25T00:32:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.545+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0ac5970 0x55f9b0ac7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.545+0000 7ff992ffd640 10 client.15573 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.545+0000 7ff9b37fe640 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0ac5970 0x55f9b0ac7d90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcaa7f584875e7cc server_cookie=2ab972dc3f6276b7 in_seq=0 out_seq=0 2024-04-25T00:32:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.560+0000 7ff992ffd640 1 -- 172.21.15.31:0/3896975217 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9b09b4540 con 0x55f9b0ac5970 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.560+0000 7ff992ffd640 10 client.15573 handle_command_reply: tid=0 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.560+0000 7ff9bcbb9740 1 client.15573 shutdown 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 msgr2=0x55f9b0a58e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a58e40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0117a0 tx=0x7ff9ac004a40 comp rx=0 tx=0).stop 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 shutdown_connections 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f9b0ac5970 0x55f9b0ac7d90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b0a4e9b0 0x55f9b0a58e40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9b0a49840 0x55f9b0a57e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 --2- 172.21.15.31:0/3896975217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9b0a48e50 0x55f9b0a578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff992ffd640 0 client.15573 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 >> 172.21.15.31:0/3896975217 conn(0x55f9b0a444c0 msgr2=0x55f9b0a47240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 shutdown_connections 2024-04-25T00:32:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 1 -- 172.21.15.31:0/3896975217 wait complete. 2024-04-25T00:32:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9bcbb9740 20 client.15573 trim_cache size 0 max 16384 2024-04-25T00:32:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:32:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:32:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:32:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff99c0a2cb0 msgr2=0x7ff99c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff99c0a2cb0 0x7ff99c0a5170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009fd0 tx=0x7ff9ac004ab0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 msgr2=0x7ff9b410b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.561+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 0x7ff9b410b020 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400d950 tx=0x7ff9a400de10 comp rx=0 tx=0).stop 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 shutdown_connections 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9b4102250 0x7ff9b4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9b4101890 0x7ff9b4109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff99c0a2cb0 0x7ff99c0a5170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 --2- 172.21.15.31:0/1480980205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9b41006a0 0x7ff9b410b020 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 >> 172.21.15.31:0/1480980205 conn(0x7ff9b40fbc10 msgr2=0x7ff9b4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 shutdown_connections 2024-04-25T00:32:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:32:54.562+0000 7ff9b9ea5640 1 -- 172.21.15.31:0/1480980205 wait complete. 2024-04-25T00:32:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:54 smithi031 ceph-mon[29167]: pgmap v431: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:54.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:54 smithi169 ceph-mon[33112]: pgmap v431: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:54.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:54 smithi040 ceph-mon[32945]: pgmap v431: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:32:56.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:56 smithi031 ceph-mon[29167]: pgmap v432: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-04-25T00:32:56.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:56 smithi169 ceph-mon[33112]: pgmap v432: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-04-25T00:32:56.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:56 smithi040 ceph-mon[32945]: pgmap v432: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-04-25T00:32:58.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:32:58 smithi031 ceph-mon[29167]: pgmap v433: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T00:32:58.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:32:58 smithi169 ceph-mon[33112]: pgmap v433: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T00:32:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:32:58 smithi040 ceph-mon[32945]: pgmap v433: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T00:33:00.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:00 smithi031 ceph-mon[29167]: pgmap v434: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-04-25T00:33:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:00 smithi169 ceph-mon[33112]: pgmap v434: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-04-25T00:33:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:00 smithi040 ceph-mon[32945]: pgmap v434: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-04-25T00:33:02.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:02 smithi031 ceph-mon[29167]: pgmap v435: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:33:02.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:02 smithi169 ceph-mon[33112]: pgmap v435: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:33:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:02 smithi040 ceph-mon[32945]: pgmap v435: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:33:04.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:04 smithi031 ceph-mon[29167]: pgmap v436: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:04 smithi169 ceph-mon[33112]: pgmap v436: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:04 smithi040 ceph-mon[32945]: pgmap v436: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:06.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:06 smithi031 ceph-mon[29167]: pgmap v437: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:06 smithi169 ceph-mon[33112]: pgmap v437: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:06 smithi040 ceph-mon[32945]: pgmap v437: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.661+0000 7f45df1ce640 1 -- 172.21.15.31:0/4255330532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 msgr2=0x7f45d8107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.661+0000 7f45df1ce640 1 --2- 172.21.15.31:0/4255330532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8107050 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009900 tx=0x7f45c402a410 comp rx=0 tx=0).stop 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.661+0000 7f45df1ce640 1 -- 172.21.15.31:0/4255330532 shutdown_connections 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.661+0000 7f45df1ce640 1 --2- 172.21.15.31:0/4255330532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d8107590 0x7f45d8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.661+0000 7f45df1ce640 1 --2- 172.21.15.31:0/4255330532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8107050 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 --2- 172.21.15.31:0/4255330532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45d8102330 0x7f45d8104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- 172.21.15.31:0/4255330532 >> 172.21.15.31:0/4255330532 conn(0x7f45d80fbcc0 msgr2=0x7f45d80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- 172.21.15.31:0/4255330532 shutdown_connections 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- 172.21.15.31:0/4255330532 wait complete. 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 Processor -- start 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- start start 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d8102330 0x7f45d8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45d8107590 0x7f45d819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810d310 con 0x7f45d8104c60 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810d190 con 0x7f45d8107590 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45df1ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810d490 con 0x7f45d8102330 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.662+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55544/0 (socket says 172.21.15.31:55544) 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 -- 172.21.15.31:0/2934673000 learned_addr learned my addr 172.21.15.31:0/2934673000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d8102330 0x7f45d8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45dd76b640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45d8107590 0x7f45d819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 -- 172.21.15.31:0/2934673000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d8102330 msgr2=0x7f45d8197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d8102330 0x7f45d8197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 -- 172.21.15.31:0/2934673000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45d8107590 msgr2=0x7f45d819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45d8107590 0x7f45d819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 -- 172.21.15.31:0/2934673000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d819f5a0 con 0x7f45d8104c60 2024-04-25T00:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d7fff640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8197ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f45c40314e0 tx=0x7f45c4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d5ffb640 1 -- 172.21.15.31:0/2934673000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c403f030 con 0x7f45d8104c60 2024-04-25T00:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d819f830 con 0x7f45d8104c60 2024-04-25T00:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d5ffb640 1 -- 172.21.15.31:0/2934673000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45c40044b0 con 0x7f45d8104c60 2024-04-25T00:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.663+0000 7f45d5ffb640 1 -- 172.21.15.31:0/2934673000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4035070 con 0x7f45d8104c60 2024-04-25T00:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.664+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d819fd70 con 0x7f45d8104c60 2024-04-25T00:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.666+0000 7f45d5ffb640 1 -- 172.21.15.31:0/2934673000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f45c4042030 con 0x7f45d8104c60 2024-04-25T00:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.666+0000 7f45e1fb9740 1 Processor -- start 2024-04-25T00:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.666+0000 7f45e1fb9740 1 -- start start 2024-04-25T00:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda694e50 0x564cda695270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda69ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564cda5f8cb0 con 0x564cda69a9b0 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564cda5a15b0 con 0x564cda695840 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564cda5f4c70 con 0x564cda694e50 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda69ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d5ffb640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45c80a2c30 0x7f45c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda69ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55558/0 (socket says 172.21.15.31:55558) 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 -- 172.21.15.31:0/3286616813 learned_addr learned my addr 172.21.15.31:0/3286616813 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45dd76b640 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda694e50 0x564cda695270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45dd76b640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45c80a2c30 0x7f45c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d5ffb640 1 -- 172.21.15.31:0/2934673000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f45c40ef560 con 0x7f45d8104c60 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 -- 172.21.15.31:0/3286616813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda694e50 msgr2=0x564cda695270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda694e50 0x564cda695270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 -- 172.21.15.31:0/3286616813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 msgr2=0x564cda69a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.667+0000 7f45d7fff640 1 -- 172.21.15.31:0/3286616813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cda30d210 con 0x564cda69a9b0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45d7fff640 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda69ce40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f45c40c4470 tx=0x7f45c40b7a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf1daf68af82e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45dd76b640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45c80a2c30 0x7f45c80a50f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x564cda6960a0 tx=0x7f45d0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45c2ffd640 1 -- 172.21.15.31:0/3286616813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4049050 con 0x564cda69a9b0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45c2ffd640 1 -- 172.21.15.31:0/3286616813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45c403b460 con 0x564cda69a9b0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45c2ffd640 1 -- 172.21.15.31:0/3286616813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40b8450 con 0x564cda69a9b0 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3286616813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 msgr2=0x564cda69ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda69ce40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f45c40c4470 tx=0x7f45c40b7a80 comp rx=0 tx=0).stop 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3286616813 shutdown_connections 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda69ce40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3286616813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda694e50 0x564cda695270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.668+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3286616813 >> 172.21.15.31:0/3286616813 conn(0x564cda6904c0 msgr2=0x564cda6908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3286616813 shutdown_connections 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3286616813 wait complete. 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 -- start start 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 Processor -- start 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda69ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda6a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564cda5f8cb0 con 0x564cda694e50 2024-04-25T00:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564cda5b4b50 con 0x564cda695840 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564cda69d720 con 0x564cda69a9b0 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.669+0000 7f45e1fb9740 10 client.0 _ll_register_callbacks cb 0x564cda5efca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda69ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda69ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55560/0 (socket says 172.21.15.31:55560) 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 -- 172.21.15.31:0/935565528 learned_addr learned my addr 172.21.15.31:0/935565528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45d7fff640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dd76b640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda6a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 -- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 msgr2=0x564cda6a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda6a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 -- 172.21.15.31:0/935565528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 msgr2=0x564cda69f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 -- 172.21.15.31:0/935565528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cda3d9620 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda69ef10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f45cc012700 tx=0x7f45cc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2045e8311a06d591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45b7fff640 1 -- 172.21.15.31:0/935565528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00f8c0 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45b7fff640 1 -- 172.21.15.31:0/935565528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f45cc00fa60 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564cda6061f0 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45b7fff640 1 -- 172.21.15.31:0/935565528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45cc00fc60 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45e1fb9740 10 client.15585 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.670+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cda6a98d0 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45b7fff640 1 -- 172.21.15.31:0/935565528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564cda6061f0 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45b7fff640 1 -- 172.21.15.31:0/935565528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45cc010430 con 0x564cda694e50 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45e1fb9740 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45e1fb9740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45e1fb9740 20 client.15585 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda6aafc0 0x564cda6ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45e1fb9740 4 client.15585 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cda600540 con 0x564cda6aafc0 2024-04-25T00:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.671+0000 7f45d7fff640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda6aafc0 0x564cda6ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45b7fff640 10 client.15585 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45d7fff640 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda6aafc0 0x564cda6ad410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7619d4095afbf5e server_cookie=6ca23e4b6cc85c00 in_seq=0 out_seq=0 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45b7fff640 1 -- 172.21.15.31:0/935565528 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cda600540 con 0x564cda6aafc0 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45b7fff640 10 client.15585 handle_command_reply: tid=0 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 client.15585 shutdown 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 msgr2=0x564cda69ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda69ef10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f45cc012700 tx=0x7f45cc012bc0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 shutdown_connections 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda6aafc0 0x564cda6ad410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda6a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda69f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/935565528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda69ef10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45b7fff640 0 client.15585 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.672+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 >> 172.21.15.31:0/935565528 conn(0x564cda6904c0 msgr2=0x564cda68f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.673+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 shutdown_connections 2024-04-25T00:33:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.673+0000 7f45e1fb9740 1 -- 172.21.15.31:0/935565528 wait complete. 2024-04-25T00:33:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.673+0000 7f45e1fb9740 20 client.15585 trim_cache size 0 max 16384 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 Processor -- start 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 -- start start 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda68ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda68f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda69e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a8c70 con 0x564cda694e50 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81aa7b0 con 0x564cda695840 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a9340 con 0x564cda69a9b0 2024-04-25T00:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda68ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda68ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55568/0 (socket says 172.21.15.31:55568) 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 -- 172.21.15.31:0/3667463427 learned_addr learned my addr 172.21.15.31:0/3667463427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dd76b640 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda69e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 -- 172.21.15.31:0/3667463427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 msgr2=0x564cda69e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda69e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 -- 172.21.15.31:0/3667463427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 msgr2=0x564cda68f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda68f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 -- 172.21.15.31:0/3667463427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cda691230 con 0x564cda694e50 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dd76b640 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda69e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda68ed30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0108a0 tx=0x7f45cc009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d9f78cd8b027e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.688+0000 7f45c27fc640 1 -- 172.21.15.31:0/3667463427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc02b070 con 0x564cda694e50 2024-04-25T00:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45c27fc640 1 -- 172.21.15.31:0/3667463427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45cc0104f0 con 0x564cda694e50 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45c27fc640 1 -- 172.21.15.31:0/3667463427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc005490 con 0x564cda694e50 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3667463427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 msgr2=0x564cda68ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda68ed30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0108a0 tx=0x7f45cc009f20 comp rx=0 tx=0).stop 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3667463427 shutdown_connections 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda69a9b0 0x564cda69e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda695840 0x564cda68f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/3667463427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda694e50 0x564cda68ed30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3667463427 >> 172.21.15.31:0/3667463427 conn(0x564cda6904c0 msgr2=0x564cda6908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3667463427 shutdown_connections 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 -- 172.21.15.31:0/3667463427 wait complete. 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.689+0000 7f45e1fb9740 1 -- start start 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 Processor -- start 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda694e50 0x564cda6a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda695840 0x564cda6a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda6a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81aa0e0 con 0x564cda69a9b0 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a9a10 con 0x564cda694e50 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a6c50 con 0x564cda695840 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45e1fb9740 10 client.0 _ll_register_callbacks cb 0x7f45cc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda6a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda6a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55582/0 (socket says 172.21.15.31:55582) 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 -- 172.21.15.31:0/2855233828 learned_addr learned my addr 172.21.15.31:0/2855233828 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45d7fff640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda695840 0x564cda6a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda694e50 0x564cda6a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 -- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda695840 msgr2=0x564cda6a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda695840 0x564cda6a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 -- 172.21.15.31:0/2855233828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda694e50 msgr2=0x564cda6a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.690+0000 7f45dd76b640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda694e50 0x564cda6a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45dd76b640 1 -- 172.21.15.31:0/2855233828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cda6a49b0 con 0x564cda69a9b0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45dd76b640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda6a43c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ae00 tx=0x7f45d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f057756c3a61fde2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45b7fff640 1 -- 172.21.15.31:0/2855233828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004b20 con 0x564cda69a9b0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45b7fff640 1 -- 172.21.15.31:0/2855233828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f45d001b070 con 0x564cda69a9b0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45cc0104f0 con 0x564cda69a9b0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45b7fff640 1 -- 172.21.15.31:0/2855233828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45d0004cc0 con 0x564cda69a9b0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45e1fb9740 10 client.15591 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cda6a5d30 con 0x564cda69a9b0 2024-04-25T00:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45b7fff640 1 -- 172.21.15.31:0/2855233828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45cc0104f0 con 0x564cda69a9b0 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45b7fff640 1 -- 172.21.15.31:0/2855233828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45d000f040 con 0x564cda69a9b0 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.691+0000 7f45e1fb9740 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45e1fb9740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45e1fb9740 20 client.15591 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda711e40 0x564cda714260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45e1fb9740 4 client.15591 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cda57e8e0 con 0x564cda711e40 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda711e40 0x564cda714260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45dcf6a640 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda711e40 0x564cda714260 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3989db17acf75b42 server_cookie=333ac97ee41af38f in_seq=0 out_seq=0 2024-04-25T00:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.692+0000 7f45b7fff640 10 client.15591 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.707+0000 7f45b7fff640 1 -- 172.21.15.31:0/2855233828 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cda57e8e0 con 0x564cda711e40 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.707+0000 7f45b7fff640 10 client.15591 handle_command_reply: tid=0 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.707+0000 7f45e1fb9740 1 client.15591 shutdown 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 msgr2=0x564cda6a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda6a43c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ae00 tx=0x7f45d0004a40 comp rx=0 tx=0).stop 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 shutdown_connections 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564cda711e40 0x564cda714260 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564cda69a9b0 0x564cda6a43c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cda695840 0x564cda6a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45b7fff640 0 client.15591 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 --2- 172.21.15.31:0/2855233828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564cda694e50 0x564cda6a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 >> 172.21.15.31:0/2855233828 conn(0x564cda6904c0 msgr2=0x564cda68ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 shutdown_connections 2024-04-25T00:33:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 1 -- 172.21.15.31:0/2855233828 wait complete. 2024-04-25T00:33:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45e1fb9740 20 client.15591 trim_cache size 0 max 16384 2024-04-25T00:33:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:33:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:33:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:33:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45c80a2c30 msgr2=0x7f45c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45df1ce640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45c80a2c30 0x7f45c80a50f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x564cda6960a0 tx=0x7f45d0009600 comp rx=0 tx=0).stop 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 msgr2=0x7f45d8197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.708+0000 7f45df1ce640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8197ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f45c40314e0 tx=0x7f45c4031a30 comp rx=0 tx=0).stop 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 shutdown_connections 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45d8107590 0x7f45d819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f45c80a2c30 0x7f45c80a50f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d8104c60 0x7f45d8197ec0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 --2- 172.21.15.31:0/2934673000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d8102330 0x7f45d8197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 >> 172.21.15.31:0/2934673000 conn(0x7f45d80fbcc0 msgr2=0x7f45d8102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 shutdown_connections 2024-04-25T00:33:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:07.709+0000 7f45df1ce640 1 -- 172.21.15.31:0/2934673000 wait complete. 2024-04-25T00:33:08.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:08 smithi031 ceph-mon[29167]: pgmap v438: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:08.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:08 smithi169 ceph-mon[33112]: pgmap v438: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:08 smithi040 ceph-mon[32945]: pgmap v438: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:10 smithi031 ceph-mon[29167]: pgmap v439: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:10 smithi169 ceph-mon[33112]: pgmap v439: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:10 smithi040 ceph-mon[32945]: pgmap v439: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:12.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:12 smithi031 ceph-mon[29167]: pgmap v440: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:33:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:12 smithi169 ceph-mon[33112]: pgmap v440: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:33:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:12 smithi040 ceph-mon[32945]: pgmap v440: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:33:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:14 smithi031 ceph-mon[29167]: pgmap v441: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:14.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:14 smithi169 ceph-mon[33112]: pgmap v441: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:14 smithi040 ceph-mon[32945]: pgmap v441: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:16.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:16 smithi031 ceph-mon[29167]: pgmap v442: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:16 smithi169 ceph-mon[33112]: pgmap v442: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:16 smithi040 ceph-mon[32945]: pgmap v442: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:18 smithi031 ceph-mon[29167]: pgmap v443: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:33:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:18 smithi169 ceph-mon[33112]: pgmap v443: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:33:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:18 smithi040 ceph-mon[32945]: pgmap v443: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:33:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:20 smithi031 ceph-mon[29167]: pgmap v444: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 -- 172.21.15.31:0/3579546594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680102300 msgr2=0x7fa680108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3579546594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680102300 0x7fa680108b90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa678009ca0 tx=0x7fa67802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 -- 172.21.15.31:0/3579546594 shutdown_connections 2024-04-25T00:33:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3579546594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680102300 0x7fa680108b90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3579546594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa680101940 0x7fa680101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3579546594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa680100750 0x7fa680100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 -- 172.21.15.31:0/3579546594 >> 172.21.15.31:0/3579546594 conn(0x7fa6800fbcc0 msgr2=0x7fa6800fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 -- 172.21.15.31:0/3579546594 shutdown_connections 2024-04-25T00:33:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.808+0000 7fa6850ae640 1 -- 172.21.15.31:0/3579546594 wait complete. 2024-04-25T00:33:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.809+0000 7fa6850ae640 1 Processor -- start 2024-04-25T00:33:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.809+0000 7fa6850ae640 1 -- start start 2024-04-25T00:33:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 0x7fa6801978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa680101940 0x7fa680197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa680102300 0x7fa68019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010c210 con 0x7fa680100750 2024-04-25T00:33:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010c090 con 0x7fa680101940 2024-04-25T00:33:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010c390 con 0x7fa680102300 2024-04-25T00:33:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 0x7fa6801978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 0x7fa6801978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44568/0 (socket says 172.21.15.31:44568) 2024-04-25T00:33:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3063731639 learned_addr learned my addr 172.21.15.31:0/3063731639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa677fff640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa680101940 0x7fa680197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67f59e640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa680102300 0x7fa68019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3063731639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa680102300 msgr2=0x7fa68019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa680102300 0x7fa68019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3063731639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa680101940 msgr2=0x7fa680197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa680101940 0x7fa680197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3063731639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68019f4d0 con 0x7fa680100750 2024-04-25T00:33:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 0x7fa6801978b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa67000d770 tx=0x7fa67000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67cd99640 1 -- 172.21.15.31:0/3063731639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670017070 con 0x7fa680100750 2024-04-25T00:33:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67cd99640 1 -- 172.21.15.31:0/3063731639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa670009c30 con 0x7fa680100750 2024-04-25T00:33:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68019f7c0 con 0x7fa680100750 2024-04-25T00:33:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.810+0000 7fa67cd99640 1 -- 172.21.15.31:0/3063731639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670003d00 con 0x7fa680100750 2024-04-25T00:33:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.811+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68019fd60 con 0x7fa680100750 2024-04-25T00:33:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.813+0000 7fa67cd99640 1 -- 172.21.15.31:0/3063731639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa670003ea0 con 0x7fa680100750 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.813+0000 7fa687db6740 1 Processor -- start 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.813+0000 7fa687db6740 1 -- start start 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67cd99640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa687db6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c03270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa687db6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c08340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa687db6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c0ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa687db6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556326b66cb0 con 0x556326c02e50 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa687db6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556326b0f5b0 con 0x556326c089b0 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa687db6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556326b62c70 con 0x556326c03840 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c03270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67f59e640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c03270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44584/0 (socket says 172.21.15.31:44584) 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 -- 172.21.15.31:0/415355114 learned_addr learned my addr 172.21.15.31:0/415355114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67f59e640 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c0ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67cd99640 1 -- 172.21.15.31:0/3063731639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa6700cec80 con 0x7fa680100750 2024-04-25T00:33:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa677fff640 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c08340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 -- 172.21.15.31:0/415355114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 msgr2=0x556326c08340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c08340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 -- 172.21.15.31:0/415355114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 msgr2=0x556326c0ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c0ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67ed9d640 1 -- 172.21.15.31:0/415355114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632687b210 con 0x556326c02e50 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.814+0000 7fa67f59e640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa678005e30 tx=0x7fa678005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.815+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c03270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa670009610 tx=0x7fa670097200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8b802b36ecc3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.815+0000 7fa6757fa640 1 -- 172.21.15.31:0/415355114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670020070 con 0x556326c02e50 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.815+0000 7fa6757fa640 1 -- 172.21.15.31:0/415355114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa670009de0 con 0x556326c02e50 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.815+0000 7fa6757fa640 1 -- 172.21.15.31:0/415355114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67009f860 con 0x556326c02e50 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.815+0000 7fa687db6740 1 -- 172.21.15.31:0/415355114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 msgr2=0x556326c03270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.815+0000 7fa687db6740 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c03270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa670009610 tx=0x7fa670097200 comp rx=0 tx=0).stop 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- 172.21.15.31:0/415355114 shutdown_connections 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c0ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c08340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 --2- 172.21.15.31:0/415355114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c03270 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- 172.21.15.31:0/415355114 >> 172.21.15.31:0/415355114 conn(0x556326bfe4c0 msgr2=0x556326bfe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- 172.21.15.31:0/415355114 shutdown_connections 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- 172.21.15.31:0/415355114 wait complete. 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- start start 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 Processor -- start 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c02e50 0x556326c0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c0d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c14860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556326c0b900 con 0x556326c089b0 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556326b22b50 con 0x556326c02e50 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556326c0ba80 con 0x556326c03840 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa687db6740 10 client.0 _ll_register_callbacks cb 0x556326b5dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa67f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c14860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa67f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c14860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44598/0 (socket says 172.21.15.31:44598) 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.816+0000 7fa67f59e640 1 -- 172.21.15.31:0/4260005057 learned_addr learned my addr 172.21.15.31:0/4260005057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa677fff640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c0d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c02e50 0x556326c0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67f59e640 1 -- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 msgr2=0x556326c0d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67f59e640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c0d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67f59e640 1 -- 172.21.15.31:0/4260005057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c02e50 msgr2=0x556326c0d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67f59e640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c02e50 0x556326c0d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67f59e640 1 -- 172.21.15.31:0/4260005057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556326947620 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa67f59e640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c14860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa678004af0 tx=0x7fa67802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35166885d02f69e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa6667fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa6667fc640 1 -- 172.21.15.31:0/4260005057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678004620 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa6667fc640 1 -- 172.21.15.31:0/4260005057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa6780047c0 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa670009de0 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa6667fc640 1 -- 172.21.15.31:0/4260005057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6780305c0 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa687db6740 10 client.15603 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.817+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556326c17bd0 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa6667fc640 1 -- 172.21.15.31:0/4260005057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa670009de0 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa6667fc640 1 -- 172.21.15.31:0/4260005057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa67804b070 con 0x556326c089b0 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa687db6740 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa687db6740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa687db6740 20 client.15603 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa687db6740 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c19360 0x556326c1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa687db6740 4 client.15603 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556326b741f0 con 0x556326c19360 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c19360 0x556326c1b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c19360 0x556326c1b7b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a802e1d50a680019 server_cookie=1d6c67d97a7ab134 in_seq=0 out_seq=0 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.818+0000 7fa6667fc640 10 client.15603 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa6667fc640 1 -- 172.21.15.31:0/4260005057 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556326b741f0 con 0x556326c19360 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa6667fc640 10 client.15603 handle_command_reply: tid=0 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 client.15603 shutdown 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 msgr2=0x556326c14860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c14860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa678004af0 tx=0x7fa67802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 shutdown_connections 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c19360 0x556326c1b7b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c14860 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c0d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa6667fc640 0 client.15603 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 --2- 172.21.15.31:0/4260005057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c02e50 0x556326c0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.819+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 >> 172.21.15.31:0/4260005057 conn(0x556326bfe4c0 msgr2=0x556326bfd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.820+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 shutdown_connections 2024-04-25T00:33:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.820+0000 7fa687db6740 1 -- 172.21.15.31:0/4260005057 wait complete. 2024-04-25T00:33:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.820+0000 7fa687db6740 20 client.15603 trim_cache size 0 max 16384 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.834+0000 7fa687db6740 1 Processor -- start 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.834+0000 7fa687db6740 1 -- start start 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa687db6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c01470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa687db6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa687db6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa687db6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a8c40 con 0x556326c02e50 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa687db6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801aa780 con 0x556326c089b0 2024-04-25T00:33:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa687db6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a9310 con 0x556326c03840 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c01470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c01470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44614/0 (socket says 172.21.15.31:44614) 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3392362792 learned_addr learned my addr 172.21.15.31:0/3392362792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa677fff640 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3392362792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 msgr2=0x556326c14820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c14820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3392362792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 msgr2=0x556326c019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 -- 172.21.15.31:0/3392362792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556326bff230 con 0x556326c02e50 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa677fff640 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.835+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c01470 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa6700979c0 tx=0x7fa670005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6272dc3abbe745a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa674ff9640 1 -- 172.21.15.31:0/3392362792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670020070 con 0x556326c02e50 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa674ff9640 1 -- 172.21.15.31:0/3392362792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa670015110 con 0x556326c02e50 2024-04-25T00:33:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa674ff9640 1 -- 172.21.15.31:0/3392362792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700af9d0 con 0x556326c02e50 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 -- 172.21.15.31:0/3392362792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 msgr2=0x556326c01470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c01470 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa6700979c0 tx=0x7fa670005d10 comp rx=0 tx=0).stop 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 -- 172.21.15.31:0/3392362792 shutdown_connections 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c089b0 0x556326c019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c03840 0x556326c14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 --2- 172.21.15.31:0/3392362792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c02e50 0x556326c01470 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 -- 172.21.15.31:0/3392362792 >> 172.21.15.31:0/3392362792 conn(0x556326bfe4c0 msgr2=0x556326bfe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 -- 172.21.15.31:0/3392362792 shutdown_connections 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.836+0000 7fa687db6740 1 -- 172.21.15.31:0/3392362792 wait complete. 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 -- start start 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 Processor -- start 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c02e50 0x556326c11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c03840 0x556326c124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c13500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801aa0b0 con 0x556326c089b0 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a99e0 con 0x556326c03840 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a6c20 con 0x556326c02e50 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa687db6740 10 client.0 _ll_register_callbacks cb 0x7fa678003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c13500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c13500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44624/0 (socket says 172.21.15.31:44624) 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 -- 172.21.15.31:0/3718443822 learned_addr learned my addr 172.21.15.31:0/3718443822 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c02e50 0x556326c11fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa677fff640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c03840 0x556326c124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 -- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c02e50 msgr2=0x556326c11fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c02e50 0x556326c11fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 -- 172.21.15.31:0/3718443822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c03840 msgr2=0x556326c124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c03840 0x556326c124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67f59e640 1 -- 172.21.15.31:0/3718443822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556326c13af0 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa677fff640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c03840 0x556326c124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.837+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c02e50 0x556326c11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa67f59e640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c13500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa678031970 tx=0x7fa67802fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0398522ffe06cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa6667fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa6667fc640 1 -- 172.21.15.31:0/3718443822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678004830 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556326b6e540 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa6667fc640 1 -- 172.21.15.31:0/3718443822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa67802f4c0 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa6667fc640 1 -- 172.21.15.31:0/3718443822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa67802f6c0 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 10 client.15609 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556326c7f930 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa6667fc640 1 -- 172.21.15.31:0/3718443822 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556326b6e540 con 0x556326c089b0 2024-04-25T00:33:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa6667fc640 1 -- 172.21.15.31:0/3718443822 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa67806a430 con 0x556326c089b0 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 10 client.15609 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 20 client.15609 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c80210 0x556326c82630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 4 client.15609 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.838+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556326aec8e0 con 0x556326c80210 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.839+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c80210 0x556326c82630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.839+0000 7fa67ed9d640 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c80210 0x556326c82630 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f954160419963b0a server_cookie=7b0978e1699c296 in_seq=0 out_seq=0 2024-04-25T00:33:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.839+0000 7fa6667fc640 10 client.15609 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa6667fc640 1 -- 172.21.15.31:0/3718443822 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556326aec8e0 con 0x556326c80210 2024-04-25T00:33:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa6667fc640 10 client.15609 handle_command_reply: tid=0 2024-04-25T00:33:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 client.15609 shutdown 2024-04-25T00:33:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 msgr2=0x556326c13500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c13500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa678031970 tx=0x7fa67802fd80 comp rx=0 tx=0).stop 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 shutdown_connections 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556326c80210 0x556326c82630 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556326c089b0 0x556326c13500 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556326c03840 0x556326c124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 --2- 172.21.15.31:0/3718443822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556326c02e50 0x556326c11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa6667fc640 0 client.15609 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 >> 172.21.15.31:0/3718443822 conn(0x556326bfe4c0 msgr2=0x556326c01470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 shutdown_connections 2024-04-25T00:33:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 1 -- 172.21.15.31:0/3718443822 wait complete. 2024-04-25T00:33:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.854+0000 7fa687db6740 20 client.15609 trim_cache size 0 max 16384 2024-04-25T00:33:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:33:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:33:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:33:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:33:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa66c0a2cb0 msgr2=0x7fa66c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa678005e30 tx=0x7fa678005f30 comp rx=0 tx=0).stop 2024-04-25T00:33:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 msgr2=0x7fa6801978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 0x7fa6801978b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa67000d770 tx=0x7fa67000dc30 comp rx=0 tx=0).stop 2024-04-25T00:33:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 shutdown_connections 2024-04-25T00:33:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa680102300 0x7fa68019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa680101940 0x7fa680197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 --2- 172.21.15.31:0/3063731639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa680100750 0x7fa6801978b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 >> 172.21.15.31:0/3063731639 conn(0x7fa6800fbcc0 msgr2=0x7fa6800fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 shutdown_connections 2024-04-25T00:33:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:20.855+0000 7fa6850ae640 1 -- 172.21.15.31:0/3063731639 wait complete. 2024-04-25T00:33:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:20 smithi169 ceph-mon[33112]: pgmap v444: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:20 smithi040 ceph-mon[32945]: pgmap v444: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:22 smithi031 ceph-mon[29167]: pgmap v445: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:22 smithi169 ceph-mon[33112]: pgmap v445: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:22 smithi040 ceph-mon[32945]: pgmap v445: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:24 smithi031 ceph-mon[29167]: pgmap v446: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:33:24.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:24 smithi169 ceph-mon[33112]: pgmap v446: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:33:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:24 smithi040 ceph-mon[32945]: pgmap v446: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:33:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:26 smithi031 ceph-mon[29167]: pgmap v447: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:33:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:26 smithi169 ceph-mon[33112]: pgmap v447: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:33:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:26 smithi040 ceph-mon[32945]: pgmap v447: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:33:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:28 smithi169 ceph-mon[33112]: pgmap v448: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:28 smithi040 ceph-mon[32945]: pgmap v448: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:28 smithi031 ceph-mon[29167]: pgmap v448: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:30 smithi169 ceph-mon[33112]: pgmap v449: 97 pgs: 97 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:33:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:30 smithi040 ceph-mon[32945]: pgmap v449: 97 pgs: 97 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:33:31.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:30 smithi031 ceph-mon[29167]: pgmap v449: 97 pgs: 97 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:33:32.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:32 smithi169 ceph-mon[33112]: pgmap v450: 97 pgs: 97 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:32 smithi040 ceph-mon[32945]: pgmap v450: 97 pgs: 97 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:33.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:32 smithi031 ceph-mon[29167]: pgmap v450: 97 pgs: 97 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:33:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 -- 172.21.15.31:0/1633374526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c1006c0 msgr2=0x7f086c100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 --2- 172.21.15.31:0/1633374526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c1006c0 0x7f086c100ae0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c02a410 comp rx=0 tx=0).stop 2024-04-25T00:33:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 -- 172.21.15.31:0/1633374526 shutdown_connections 2024-04-25T00:33:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 --2- 172.21.15.31:0/1633374526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f086c102270 0x7f086c108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 --2- 172.21.15.31:0/1633374526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c1018b0 0x7f086c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 --2- 172.21.15.31:0/1633374526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c1006c0 0x7f086c100ae0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 -- 172.21.15.31:0/1633374526 >> 172.21.15.31:0/1633374526 conn(0x7f086c0fbc50 msgr2=0x7f086c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 -- 172.21.15.31:0/1633374526 shutdown_connections 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.954+0000 7f087235c640 1 -- 172.21.15.31:0/1633374526 wait complete. 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 Processor -- start 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 -- start start 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f086c1006c0 0x7f086c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c1018b0 0x7f086c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 0x7f086c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10c1c0 con 0x7f086c102270 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10c040 con 0x7f086c1006c0 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f087235c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10c340 con 0x7f086c1018b0 2024-04-25T00:33:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f08708f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 0x7f086c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f08708f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 0x7f086c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39118/0 (socket says 172.21.15.31:39118) 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f08708f9640 1 -- 172.21.15.31:0/2636646831 learned_addr learned my addr 172.21.15.31:0/2636646831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f086bfff640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f086c1006c0 0x7f086c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.955+0000 7f086b7fe640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c1018b0 0x7f086c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08708f9640 1 -- 172.21.15.31:0/2636646831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c1018b0 msgr2=0x7f086c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08708f9640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c1018b0 0x7f086c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08708f9640 1 -- 172.21.15.31:0/2636646831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f086c1006c0 msgr2=0x7f086c197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08708f9640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f086c1006c0 0x7f086c197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08708f9640 1 -- 172.21.15.31:0/2636646831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c19f520 con 0x7f086c102270 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08708f9640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 0x7f086c19eeb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f086400cce0 tx=0x7f086400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08697fa640 1 -- 172.21.15.31:0/2636646831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086400f450 con 0x7f086c102270 2024-04-25T00:33:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08697fa640 1 -- 172.21.15.31:0/2636646831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086400faa0 con 0x7f086c102270 2024-04-25T00:33:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c19f810 con 0x7f086c102270 2024-04-25T00:33:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f08697fa640 1 -- 172.21.15.31:0/2636646831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864018b70 con 0x7f086c102270 2024-04-25T00:33:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.956+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c19fd10 con 0x7f086c102270 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.958+0000 7f08697fa640 1 -- 172.21.15.31:0/2636646831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0864018d10 con 0x7f086c102270 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 Processor -- start 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 -- start start 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74df840 0x55ade74e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ade7442cb0 con 0x55ade74dee50 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ade73eb5b0 con 0x55ade74e49b0 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f0875060740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ade743ec70 con 0x55ade74df840 2024-04-25T00:33:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f08697fa640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f08540a2c30 0x7f08540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f086b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f086b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74df270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39124/0 (socket says 172.21.15.31:39124) 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f086b7fe640 1 -- 172.21.15.31:0/1168389578 learned_addr learned my addr 172.21.15.31:0/1168389578 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f086bfff640 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74df840 0x55ade74e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f08708f9640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f08540a2c30 0x7f08540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.959+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f086b7fe640 1 -- 172.21.15.31:0/1168389578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74df840 msgr2=0x55ade74e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74df840 0x55ade74e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f08697fa640 1 -- 172.21.15.31:0/2636646831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f08640cf4b0 con 0x7f086c102270 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f086b7fe640 1 -- 172.21.15.31:0/1168389578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 msgr2=0x55ade74e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f086b7fe640 1 -- 172.21.15.31:0/1168389578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ade7157210 con 0x55ade74dee50 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74df270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f086000bfa0 tx=0x7f086000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61af5310f433b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f08527fc640 1 -- 172.21.15.31:0/1168389578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004030 con 0x55ade74dee50 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f08708f9640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f08540a2c30 0x7f08540a50f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f086409bc10 tx=0x7f0864066900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f08527fc640 1 -- 172.21.15.31:0/1168389578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08600041d0 con 0x55ade74dee50 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.960+0000 7f08527fc640 1 -- 172.21.15.31:0/1168389578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860010460 con 0x55ade74dee50 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 -- 172.21.15.31:0/1168389578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 msgr2=0x55ade74df270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74df270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f086000bfa0 tx=0x7f086000b840 comp rx=0 tx=0).stop 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 -- 172.21.15.31:0/1168389578 shutdown_connections 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74df840 0x55ade74e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 --2- 172.21.15.31:0/1168389578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74df270 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 -- 172.21.15.31:0/1168389578 >> 172.21.15.31:0/1168389578 conn(0x55ade74da4c0 msgr2=0x55ade74da8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 -- 172.21.15.31:0/1168389578 shutdown_connections 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 -- 172.21.15.31:0/1168389578 wait complete. 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 -- start start 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.961+0000 7f0875060740 1 Processor -- start 2024-04-25T00:33:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ade74e7960 con 0x55ade74df840 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ade73feb50 con 0x55ade74e49b0 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ade74e7ae0 con 0x55ade74dee50 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f0875060740 10 client.0 _ll_register_callbacks cb 0x55ade7439ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39132/0 (socket says 172.21.15.31:39132) 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 -- 172.21.15.31:0/48958900 learned_addr learned my addr 172.21.15.31:0/48958900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086b7fe640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086b7fe640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 -- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74dee50 msgr2=0x55ade74e9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 -- 172.21.15.31:0/48958900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 msgr2=0x55ade74f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 -- 172.21.15.31:0/48958900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ade7223620 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086b7fe640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f086bfff640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74e97b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f086c101710 tx=0x7f085c0315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1aecf7e510776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.962+0000 7f084b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f084b7fe640 1 -- 172.21.15.31:0/48958900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c035040 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f084b7fe640 1 -- 172.21.15.31:0/48958900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f085c004240 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08600041d0 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f084b7fe640 1 -- 172.21.15.31:0/48958900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f085c004440 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 10 client.15621 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ade74f3bd0 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f084b7fe640 1 -- 172.21.15.31:0/48958900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08600041d0 con 0x55ade74df840 2024-04-25T00:33:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f084b7fe640 1 -- 172.21.15.31:0/48958900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f085c04f070 con 0x55ade74df840 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 10 client.15621 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 20 client.15621 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade74f5310 0x55ade74f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 4 client.15621 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.963+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ade74501f0 con 0x55ade74f5310 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.964+0000 7f086b7fe640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade74f5310 0x55ade74f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.964+0000 7f086b7fe640 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade74f5310 0x55ade74f7750 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e496939ca1137d59 server_cookie=b731304b1ff11fc7 in_seq=0 out_seq=0 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.964+0000 7f084b7fe640 10 client.15621 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f084b7fe640 1 -- 172.21.15.31:0/48958900 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ade74501f0 con 0x55ade74f5310 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f084b7fe640 10 client.15621 handle_command_reply: tid=0 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 client.15621 shutdown 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 msgr2=0x55ade74e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74e97b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f086c101710 tx=0x7f085c0315a0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 shutdown_connections 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade74f5310 0x55ade74f7750 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74e49b0 0x55ade74f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f084b7fe640 0 client.15621 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 --2- 172.21.15.31:0/48958900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 >> 172.21.15.31:0/48958900 conn(0x55ade74da4c0 msgr2=0x55ade74d95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 shutdown_connections 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 1 -- 172.21.15.31:0/48958900 wait complete. 2024-04-25T00:33:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.965+0000 7f0875060740 20 client.15621 trim_cache size 0 max 16384 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.980+0000 7f0875060740 1 Processor -- start 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 -- start start 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade74f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a8ac0 con 0x55ade74dee50 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1aa600 con 0x55ade74df840 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0875060740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a9190 con 0x55ade74e49b0 2024-04-25T00:33:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39142/0 (socket says 172.21.15.31:39142) 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 -- 172.21.15.31:0/4026923713 learned_addr learned my addr 172.21.15.31:0/4026923713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade74f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 -- 172.21.15.31:0/4026923713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 msgr2=0x55ade74f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade74f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 -- 172.21.15.31:0/4026923713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74df840 msgr2=0x55ade74e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 -- 172.21.15.31:0/4026923713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ade74ea3f0 con 0x55ade74dee50 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f086b7fe640 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74e9270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0860015010 tx=0x7f086000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaf5f2506643118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.981+0000 7f0851ffb640 1 -- 172.21.15.31:0/4026923713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860032070 con 0x55ade74dee50 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0851ffb640 1 -- 172.21.15.31:0/4026923713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086002c3b0 con 0x55ade74dee50 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0851ffb640 1 -- 172.21.15.31:0/4026923713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860003850 con 0x55ade74dee50 2024-04-25T00:33:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 -- 172.21.15.31:0/4026923713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 msgr2=0x55ade74e9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74e9270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0860015010 tx=0x7f086000bfd0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 -- 172.21.15.31:0/4026923713 shutdown_connections 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade74f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74df840 0x55ade74e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 --2- 172.21.15.31:0/4026923713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74dee50 0x55ade74e9270 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 -- 172.21.15.31:0/4026923713 >> 172.21.15.31:0/4026923713 conn(0x55ade74da4c0 msgr2=0x55ade74da8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 -- 172.21.15.31:0/4026923713 shutdown_connections 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 -- 172.21.15.31:0/4026923713 wait complete. 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.982+0000 7f0875060740 1 -- start start 2024-04-25T00:33:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 Processor -- start 2024-04-25T00:33:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74dee50 0x55ade74ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade75598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a9f30 con 0x55ade74df840 2024-04-25T00:33:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a9860 con 0x55ade74dee50 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a6aa0 con 0x55ade74e49b0 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f0875060740 10 client.0 _ll_register_callbacks cb 0x7f085c031cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39150/0 (socket says 172.21.15.31:39150) 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 -- 172.21.15.31:0/1275096829 learned_addr learned my addr 172.21.15.31:0/1275096829 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74dee50 0x55ade74ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade75598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 -- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 msgr2=0x55ade75598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade75598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 -- 172.21.15.31:0/1275096829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74dee50 msgr2=0x55ade74ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74dee50 0x55ade74ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.983+0000 7f086bfff640 1 -- 172.21.15.31:0/1275096829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ade7559e00 con 0x55ade74df840 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f086bfff640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74edcc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55ade74ea750 tx=0x7f085c046cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8713c4102941c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f084b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f084b7fe640 1 -- 172.21.15.31:0/1275096829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c055020 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f084b7fe640 1 -- 172.21.15.31:0/1275096829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f085c0518a0 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086002c3b0 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f084b7fe640 1 -- 172.21.15.31:0/1275096829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f085c060050 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 10 client.15627 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ade755b130 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f084b7fe640 1 -- 172.21.15.31:0/1275096829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086002c3b0 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f084b7fe640 1 -- 172.21.15.31:0/1275096829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f085c04f040 con 0x55ade74df840 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 10 client.15627 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 20 client.15627 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade755bb30 0x55ade755df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 4 client.15627 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.984+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ade744a540 con 0x55ade755bb30 2024-04-25T00:33:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.985+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade755bb30 0x55ade755df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.985+0000 7f084b7fe640 10 client.15627 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:33.985+0000 7f086b7fe640 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade755bb30 0x55ade755df50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3579075963bfa685 server_cookie=80bcae6dc3ce408 in_seq=0 out_seq=0 2024-04-25T00:33:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f084b7fe640 1 -- 172.21.15.31:0/1275096829 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ade744a540 con 0x55ade755bb30 2024-04-25T00:33:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f084b7fe640 10 client.15627 handle_command_reply: tid=0 2024-04-25T00:33:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 client.15627 shutdown 2024-04-25T00:33:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 msgr2=0x55ade74edcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74edcc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55ade74ea750 tx=0x7f085c046cc0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 shutdown_connections 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55ade755bb30 0x55ade755df50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ade74e49b0 0x55ade75598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ade74df840 0x55ade74edcc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 --2- 172.21.15.31:0/1275096829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ade74dee50 0x55ade74ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f084b7fe640 0 client.15627 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.001+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 >> 172.21.15.31:0/1275096829 conn(0x55ade74da4c0 msgr2=0x55ade74e9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 shutdown_connections 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f0875060740 1 -- 172.21.15.31:0/1275096829 wait complete. 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f0875060740 20 client.15627 trim_cache size 0 max 16384 2024-04-25T00:33:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:33:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:33:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:33:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:33:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f08540a2c30 msgr2=0x7f08540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f08540a2c30 0x7f08540a50f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f086409bc10 tx=0x7f0864066900 comp rx=0 tx=0).stop 2024-04-25T00:33:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 msgr2=0x7f086c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 0x7f086c19eeb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f086400cce0 tx=0x7f086400a620 comp rx=0 tx=0).stop 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 shutdown_connections 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f08540a2c30 0x7f08540a50f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c102270 0x7f086c19eeb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c1018b0 0x7f086c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 --2- 172.21.15.31:0/2636646831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f086c1006c0 0x7f086c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 >> 172.21.15.31:0/2636646831 conn(0x7f086c0fbc50 msgr2=0x7f086c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 shutdown_connections 2024-04-25T00:33:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:34.002+0000 7f087235c640 1 -- 172.21.15.31:0/2636646831 wait complete. 2024-04-25T00:33:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:34 smithi169 ceph-mon[33112]: pgmap v451: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:34 smithi040 ceph-mon[32945]: pgmap v451: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:34 smithi031 ceph-mon[29167]: pgmap v451: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:36 smithi169 ceph-mon[33112]: pgmap v452: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:33:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:36 smithi040 ceph-mon[32945]: pgmap v452: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:33:37.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:36 smithi031 ceph-mon[29167]: pgmap v452: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:33:38.824 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:38 smithi031 ceph-mon[29167]: pgmap v453: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:33:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:38 smithi169 ceph-mon[33112]: pgmap v453: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:33:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:38 smithi040 ceph-mon[32945]: pgmap v453: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:33:40.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:40 smithi169 ceph-mon[33112]: pgmap v454: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:40 smithi040 ceph-mon[32945]: pgmap v454: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:41.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:40 smithi031 ceph-mon[29167]: pgmap v454: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:33:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:42 smithi169 ceph-mon[33112]: pgmap v455: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:42 smithi040 ceph-mon[32945]: pgmap v455: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:43.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:42 smithi031 ceph-mon[29167]: pgmap v455: 97 pgs: 97 active+clean; 1.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:44 smithi169 ceph-mon[33112]: pgmap v456: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:33:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:44 smithi040 ceph-mon[32945]: pgmap v456: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:33:45.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:44 smithi031 ceph-mon[29167]: pgmap v456: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:33:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:46 smithi169 ceph-mon[33112]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:46 smithi040 ceph-mon[32945]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:46 smithi031 ceph-mon[29167]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 -- 172.21.15.31:0/2393198630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4101890 msgr2=0x7f59c4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/2393198630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4101890 0x7f59c4101d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009900 tx=0x7f59c002a410 comp rx=0 tx=0).stop 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 -- 172.21.15.31:0/2393198630 shutdown_connections 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/2393198630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59c4102250 0x7f59c4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/2393198630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4101890 0x7f59c4101d10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/2393198630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 0x7f59c4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 -- 172.21.15.31:0/2393198630 >> 172.21.15.31:0/2393198630 conn(0x7f59c40fbc10 msgr2=0x7f59c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 -- 172.21.15.31:0/2393198630 shutdown_connections 2024-04-25T00:33:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.102+0000 7f59cb8a5640 1 -- 172.21.15.31:0/2393198630 wait complete. 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 Processor -- start 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 -- start start 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 0x7f59c4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59c4101890 0x7f59c4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 0x7f59c419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410c1b0 con 0x7f59c4102250 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410c030 con 0x7f59c41006a0 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59cb8a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410c330 con 0x7f59c4101890 2024-04-25T00:33:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59c9e42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 0x7f59c419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59c9641640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 0x7f59c4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59c9e42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 0x7f59c419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56972/0 (socket says 172.21.15.31:56972) 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59c9e42640 1 -- 172.21.15.31:0/830327713 learned_addr learned my addr 172.21.15.31:0/830327713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.103+0000 7f59c8e40640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59c4101890 0x7f59c4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9e42640 1 -- 172.21.15.31:0/830327713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59c4101890 msgr2=0x7f59c4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9e42640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59c4101890 0x7f59c4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9e42640 1 -- 172.21.15.31:0/830327713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 msgr2=0x7f59c4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9e42640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 0x7f59c4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9e42640 1 -- 172.21.15.31:0/830327713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c419f560 con 0x7f59c4102250 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9641640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 0x7f59c4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59c9e42640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 0x7f59c419eef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00cce0 tx=0x7f59bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59ba7fc640 1 -- 172.21.15.31:0/830327713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00f450 con 0x7f59c4102250 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59ba7fc640 1 -- 172.21.15.31:0/830327713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59bc00faa0 con 0x7f59c4102250 2024-04-25T00:33:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59ba7fc640 1 -- 172.21.15.31:0/830327713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc018b70 con 0x7f59c4102250 2024-04-25T00:33:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c419f850 con 0x7f59c4102250 2024-04-25T00:33:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.104+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c419fd70 con 0x7f59c4102250 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.106+0000 7f59ba7fc640 1 -- 172.21.15.31:0/830327713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f59bc00fc50 con 0x7f59c4102250 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 Processor -- start 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 -- start start 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fdc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640fe1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fe4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557640f3bcb0 con 0x557640fdc260 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557640ee45b0 con 0x557640fdcc50 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557640f37c70 con 0x557640fe1dc0 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59ba7fc640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f59b00a2c30 0x7f59b00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59c8e40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fdc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59c8e40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fdc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56974/0 (socket says 172.21.15.31:56974) 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59c8e40640 1 -- 172.21.15.31:0/2271595636 learned_addr learned my addr 172.21.15.31:0/2271595636 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.107+0000 7f59c9641640 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640fe1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fe4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c9e42640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f59b00a2c30 0x7f59b00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59ba7fc640 1 -- 172.21.15.31:0/830327713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f59bc0cf3d0 con 0x7f59c4102250 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 -- 172.21.15.31:0/2271595636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 msgr2=0x557640fe4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fe4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 -- 172.21.15.31:0/2271595636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 msgr2=0x557640fe1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640fe1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 -- 172.21.15.31:0/2271595636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557640c50210 con 0x557640fdc260 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c9641640 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640fe1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c9e42640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f59b00a2c30 0x7f59b00a50f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00ccb0 tx=0x7f59bc066820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fdc680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005ec0 tx=0x7f59c0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99850933f42cae0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59af7fe640 1 -- 172.21.15.31:0/2271595636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0035030 con 0x557640fdc260 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59af7fe640 1 -- 172.21.15.31:0/2271595636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c0031760 con 0x557640fdc260 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.108+0000 7f59af7fe640 1 -- 172.21.15.31:0/2271595636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0003d60 con 0x557640fdc260 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2271595636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 msgr2=0x557640fdc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fdc680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005ec0 tx=0x7f59c0005e50 comp rx=0 tx=0).stop 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2271595636 shutdown_connections 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fe4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640fe1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2271595636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fdc680 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2271595636 >> 172.21.15.31:0/2271595636 conn(0x557640fd34c0 msgr2=0x557640fd38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2271595636 shutdown_connections 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2271595636 wait complete. 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.109+0000 7f59ce5ba740 1 -- start start 2024-04-25T00:33:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 Processor -- start 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fe6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdcc50 0x557640fe6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fe1dc0 0x557640fedc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557640fe4d10 con 0x557640fdc260 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557640ef7b50 con 0x557640fe1dc0 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557640fe4e90 con 0x557640fdcc50 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59ce5ba740 10 client.0 _ll_register_callbacks cb 0x557640f32ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fe6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fe6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56988/0 (socket says 172.21.15.31:56988) 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 -- 172.21.15.31:0/4038467992 learned_addr learned my addr 172.21.15.31:0/4038467992 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c9641640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdcc50 0x557640fe6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fe1dc0 0x557640fedc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 -- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdcc50 msgr2=0x557640fe6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdcc50 0x557640fe6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 -- 172.21.15.31:0/4038467992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fe1dc0 msgr2=0x557640fedc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fe1dc0 0x557640fedc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.110+0000 7f59c8e40640 1 -- 172.21.15.31:0/4038467992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557640d1c620 con 0x557640fdc260 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59c8e40640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fe6680 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f59c00096c0 tx=0x7f59c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b2c8fe188ac98d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59acff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59acff9640 1 -- 172.21.15.31:0/4038467992 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0035070 con 0x557640fdc260 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59acff9640 1 -- 172.21.15.31:0/4038467992 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f59c0004030 con 0x557640fdc260 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c0031760 con 0x557640fdc260 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59acff9640 1 -- 172.21.15.31:0/4038467992 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59c0004230 con 0x557640fdc260 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 10 client.15639 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557640fe4d10 con 0x557640fdc260 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59acff9640 1 -- 172.21.15.31:0/4038467992 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c0031760 con 0x557640fdc260 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59acff9640 1 -- 172.21.15.31:0/4038467992 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f59c003b460 con 0x557640fdc260 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 20 client.15639 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557640ff2220 0x557640ff4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 4 client.15639 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.111+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557640f491f0 con 0x557640ff2220 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.112+0000 7f59c9641640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557640ff2220 0x557640ff4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.112+0000 7f59acff9640 10 client.15639 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.112+0000 7f59c9641640 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557640ff2220 0x557640ff4610 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b2d7999e08a87d server_cookie=6f8795b29ecd57be in_seq=0 out_seq=0 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59acff9640 1 -- 172.21.15.31:0/4038467992 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557640f491f0 con 0x557640ff2220 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59acff9640 10 client.15639 handle_command_reply: tid=0 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 client.15639 shutdown 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 msgr2=0x557640fe6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fe6680 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f59c00096c0 tx=0x7f59c0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 shutdown_connections 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557640ff2220 0x557640ff4610 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fe1dc0 0x557640fedc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdcc50 0x557640fe6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59acff9640 0 client.15639 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/4038467992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdc260 0x557640fe6680 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 >> 172.21.15.31:0/4038467992 conn(0x557640fd34c0 msgr2=0x557640fd25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 shutdown_connections 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 1 -- 172.21.15.31:0/4038467992 wait complete. 2024-04-25T00:33:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.113+0000 7f59ce5ba740 20 client.15639 trim_cache size 0 max 16384 2024-04-25T00:33:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 Processor -- start 2024-04-25T00:33:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 -- start start 2024-04-25T00:33:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdc260 0x557640fe69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 0x557640fedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fd6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a6970 con 0x557640fdcc50 2024-04-25T00:33:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a84b0 con 0x557640fdc260 2024-04-25T00:33:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.128+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a7040 con 0x557640fe1dc0 2024-04-25T00:33:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 0x557640fedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 0x557640fedbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56998/0 (socket says 172.21.15.31:56998) 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 -- 172.21.15.31:0/2008775611 learned_addr learned my addr 172.21.15.31:0/2008775611 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fd6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdc260 0x557640fe69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 -- 172.21.15.31:0/2008775611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 msgr2=0x557640fd6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fd6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 -- 172.21.15.31:0/2008775611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdc260 msgr2=0x557640fe69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdc260 0x557640fe69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 -- 172.21.15.31:0/2008775611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557640fd3f10 con 0x557640fdcc50 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c8e40640 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdc260 0x557640fe69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59c9641640 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 0x557640fedbc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x557640fdd050 tx=0x7f59b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3e878ec7742db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59aeffd640 1 -- 172.21.15.31:0/2008775611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4012d00 con 0x557640fdcc50 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59aeffd640 1 -- 172.21.15.31:0/2008775611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b40054d0 con 0x557640fdcc50 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.129+0000 7f59aeffd640 1 -- 172.21.15.31:0/2008775611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40047a0 con 0x557640fdcc50 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2008775611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 msgr2=0x557640fedbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 0x557640fedbc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x557640fdd050 tx=0x7f59b4005f90 comp rx=0 tx=0).stop 2024-04-25T00:33:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2008775611 shutdown_connections 2024-04-25T00:33:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fe1dc0 0x557640fd6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fdcc50 0x557640fedbc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/2008775611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdc260 0x557640fe69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2008775611 >> 172.21.15.31:0/2008775611 conn(0x557640fd34c0 msgr2=0x557640fe6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2008775611 shutdown_connections 2024-04-25T00:33:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 -- 172.21.15.31:0/2008775611 wait complete. 2024-04-25T00:33:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 -- start start 2024-04-25T00:33:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.130+0000 7f59ce5ba740 1 Processor -- start 2024-04-25T00:33:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdc260 0x557640feacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640feb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 0x557640fec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a7de0 con 0x557640fe1dc0 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a7710 con 0x557640fdcc50 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a4950 con 0x557640fdc260 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59ce5ba740 10 client.0 _ll_register_callbacks cb 0x7f59b4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 0x557640fec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 0x557640fec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57008/0 (socket says 172.21.15.31:57008) 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 -- 172.21.15.31:0/549689444 learned_addr learned my addr 172.21.15.31:0/549689444 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:33:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c9641640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640feb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdc260 0x557640feacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 -- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdc260 msgr2=0x557640feacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdc260 0x557640feacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 -- 172.21.15.31:0/549689444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 msgr2=0x557640feb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640feb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 -- 172.21.15.31:0/549689444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557640fec7c0 con 0x557640fe1dc0 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c9641640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640feb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59c8e40640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 0x557640fec1d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f59c003d020 tx=0x7f59c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967900c8ea637afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.131+0000 7f59acff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59acff9640 1 -- 172.21.15.31:0/549689444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0035040 con 0x557640fe1dc0 2024-04-25T00:33:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59acff9640 1 -- 172.21.15.31:0/549689444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f59c003baf0 con 0x557640fe1dc0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b40054d0 con 0x557640fe1dc0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59acff9640 1 -- 172.21.15.31:0/549689444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59c003bcf0 con 0x557640fe1dc0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 10 client.15645 fetch_fsmap learned FSMap version 26 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557640fedab0 con 0x557640fe1dc0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59acff9640 1 -- 172.21.15.31:0/549689444 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b40054d0 con 0x557640fe1dc0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59acff9640 1 -- 172.21.15.31:0/549689444 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f59c0056a70 con 0x557640fe1dc0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 10 client.15645 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 20 client.15645 populate_metadata read hostname 'smithi031' 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557641058a20 0x55764105ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 4 client.15645 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.132+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557640f43540 con 0x557641058a20 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.133+0000 7f59c9641640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557641058a20 0x55764105ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.133+0000 7f59acff9640 10 client.15645 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.133+0000 7f59c9641640 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557641058a20 0x55764105ae40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd21e6b744e00adc server_cookie=c98a6683147a0a16 in_seq=0 out_seq=0 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59acff9640 1 -- 172.21.15.31:0/549689444 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557640f43540 con 0x557641058a20 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59acff9640 10 client.15645 handle_command_reply: tid=0 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 client.15645 shutdown 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 msgr2=0x557640fec1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 0x557640fec1d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f59c003d020 tx=0x7f59c002a410 comp rx=0 tx=0).stop 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 shutdown_connections 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x557641058a20 0x55764105ae40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557640fe1dc0 0x557640fec1d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59acff9640 0 client.15645 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557640fdcc50 0x557640feb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 --2- 172.21.15.31:0/549689444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557640fdc260 0x557640feacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 >> 172.21.15.31:0/549689444 conn(0x557640fd34c0 msgr2=0x557640fd6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 shutdown_connections 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 1 -- 172.21.15.31:0/549689444 wait complete. 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.147+0000 7f59ce5ba740 20 client.15645 trim_cache size 0 max 16384 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:33:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:33:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f59b00a2c30 msgr2=0x7f59b00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f59b00a2c30 0x7f59b00a50f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00ccb0 tx=0x7f59bc066820 comp rx=0 tx=0).stop 2024-04-25T00:33:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 msgr2=0x7f59c419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 0x7f59c419eef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00cce0 tx=0x7f59bc00a620 comp rx=0 tx=0).stop 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 shutdown_connections 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f59b00a2c30 0x7f59b00a50f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59c4102250 0x7f59c419eef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59c4101890 0x7f59c4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 --2- 172.21.15.31:0/830327713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59c41006a0 0x7f59c4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 >> 172.21.15.31:0/830327713 conn(0x7f59c40fbc10 msgr2=0x7f59c40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 shutdown_connections 2024-04-25T00:33:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:33:47.148+0000 7f59cb8a5640 1 -- 172.21.15.31:0/830327713 wait complete. 2024-04-25T00:33:48.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:48 smithi169 ceph-mon[33112]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T00:33:48.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:48 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:33:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:48 smithi040 ceph-mon[32945]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T00:33:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:48 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:33:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:48 smithi031 ceph-mon[29167]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T00:33:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:48 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:33:49.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:33:49.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:33:49.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:33:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:33:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:33:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:33:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:33:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:33:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:33:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:51 smithi031 ceph-mon[29167]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-25T00:33:51.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:51 smithi169 ceph-mon[33112]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-25T00:33:51.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:51 smithi040 ceph-mon[32945]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-25T00:33:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:53 smithi031 ceph-mon[29167]: pgmap v460: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:33:53.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:53 smithi169 ceph-mon[33112]: pgmap v460: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:33:53.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:53 smithi040 ceph-mon[32945]: pgmap v460: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:33:55.165 INFO:tasks.workunit.client.0.smithi031.stdout:64/600: open fd = 3 2024-04-25T00:33:55.165 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.165 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.165 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.165 INFO:tasks.workunit.client.0.smithi031.stdout:65/600: open fd = 3 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:66/600: open fd = 3 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:67/600: open fd = 3 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:68/600: open fd = 3 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:69/600: open fd = 3 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:70/600: open fd = 3 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:71/600: open fd = 3 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:72/600: open fd = 3 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:73/600: open fd = 3 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.168 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:74/600: open fd = 3 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:75/600: open fd = 3 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:76/600: open fd = 3 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.169 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:77/600: open fd = 3 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:78/600: open fd = 3 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.170 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:79/600: open fd = 3 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:80/600: open fd = 3 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:81/600: open fd = 3 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:82/600: open fd = 3 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:83/600: open fd = 3 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:84/600: open fd = 3 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.172 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:85/600: open fd = 3 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:86/600: open fd = 3 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:87/600: open fd = 3 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:88/600: open fd = 3 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:89/600: open fd = 3 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:90/600: open fd = 3 2024-04-25T00:33:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:91/600: open fd = 3 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:92/600: open fd = 3 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:93/600: open fd = 3 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.175 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:94/600: open fd = 3 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:95/600: open fd = 3 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:96/600: open fd = 3 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:97/600: open fd = 3 2024-04-25T00:33:55.176 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:98/600: open fd = 3 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:99/600: open fd = 3 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:100/600: open fd = 3 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:101/600: open fd = 3 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:102/600: open fd = 3 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:103/600: open fd = 3 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:104/600: open fd = 3 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:105/600: open fd = 3 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:106/600: open fd = 3 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.179 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:107/600: open fd = 3 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:108/600: open fd = 3 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:109/600: open fd = 3 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.180 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:110/600: open fd = 3 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:111/600: open fd = 3 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:112/600: open fd = 3 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.181 INFO:tasks.workunit.client.0.smithi031.stdout:113/600: open fd = 3 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:114/600: open fd = 3 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:115/600: open fd = 3 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.182 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:116/600: open fd = 3 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:117/600: open fd = 3 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:118/600: open fd = 3 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:119/600: open fd = 3 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:120/600: open fd = 3 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:121/600: open fd = 3 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:122/600: open fd = 3 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.184 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:123/600: open fd = 3 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:124/600: open fd = 3 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:125/600: open fd = 3 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:126/600: open fd = 3 2024-04-25T00:33:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:33:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:55 smithi031 ceph-mon[29167]: pgmap v461: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:55.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:55 smithi169 ceph-mon[33112]: pgmap v461: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:55.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:55 smithi040 ceph-mon[32945]: pgmap v461: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:33:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:57 smithi031 ceph-mon[29167]: pgmap v462: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:33:57.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:57 smithi169 ceph-mon[33112]: pgmap v462: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:33:57.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:57 smithi040 ceph-mon[32945]: pgmap v462: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:33:59.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:33:59 smithi031 ceph-mon[29167]: pgmap v463: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:59.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:33:59 smithi169 ceph-mon[33112]: pgmap v463: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:33:59.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:33:59 smithi040 ceph-mon[32945]: pgmap v463: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.251+0000 7f3ed49ed640 1 -- 172.21.15.31:0/4270137928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 msgr2=0x7f3ed0102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.251+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/4270137928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0102fa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009930 tx=0x7f3ec402a420 comp rx=0 tx=0).stop 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 -- 172.21.15.31:0/4270137928 shutdown_connections 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/4270137928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed01034e0 0x7f3ed0109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/4270137928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0102fa0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/4270137928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ed00ffab0 0x7f3ed00ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 -- 172.21.15.31:0/4270137928 >> 172.21.15.31:0/4270137928 conn(0x7f3ed00fb3c0 msgr2=0x7f3ed00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 -- 172.21.15.31:0/4270137928 shutdown_connections 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 -- 172.21.15.31:0/4270137928 wait complete. 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 Processor -- start 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.252+0000 7f3ed49ed640 1 -- start start 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ed49ed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ed00ffab0 0x7f3ed0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ed49ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ed49ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed01034e0 0x7f3ed0101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ed49ed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed010d2f0 con 0x7f3ed0102b20 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ed49ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed010d170 con 0x7f3ed01034e0 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ed49ed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed010d470 con 0x7f3ed00ffab0 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47816/0 (socket says 172.21.15.31:47816) 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/3370672652 learned_addr learned my addr 172.21.15.31:0/3370672652 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ece59c640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ed00ffab0 0x7f3ed0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3eced9d640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed01034e0 0x7f3ed0101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/3370672652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ed00ffab0 msgr2=0x7f3ed0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ed00ffab0 0x7f3ed0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/3370672652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed01034e0 msgr2=0x7f3ed0101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed01034e0 0x7f3ed0101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/3370672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed01018a0 con 0x7f3ed0102b20 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.253+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0100ad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40314e0 tx=0x7f3ec4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.254+0000 7f3ebf7fe640 1 -- 172.21.15.31:0/3370672652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec403f030 con 0x7f3ed0102b20 2024-04-25T00:34:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.254+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed01a1a20 con 0x7f3ed0102b20 2024-04-25T00:34:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.254+0000 7f3ebf7fe640 1 -- 172.21.15.31:0/3370672652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec40044b0 con 0x7f3ed0102b20 2024-04-25T00:34:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.254+0000 7f3ebf7fe640 1 -- 172.21.15.31:0/3370672652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4035070 con 0x7f3ed0102b20 2024-04-25T00:34:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.254+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01a1f70 con 0x7f3ed0102b20 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.256+0000 7f3ebf7fe640 1 -- 172.21.15.31:0/3370672652 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3ec4042030 con 0x7f3ed0102b20 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.256+0000 7f3ed77cf740 1 Processor -- start 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 -- start start 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172e8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c51724bcb0 con 0x55c5172e8840 2024-04-25T00:34:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c5171f45b0 con 0x55c5172e7e50 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c517247c70 con 0x55c5172ed9b0 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ebf7fe640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3eb80a2c30 0x7f3eb80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47828/0 (socket says 172.21.15.31:47828) 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 -- 172.21.15.31:0/3636034547 learned_addr learned my addr 172.21.15.31:0/3636034547 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3eced9d640 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3eced9d640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3eb80a2c30 0x7f3eb80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 -- 172.21.15.31:0/3636034547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 msgr2=0x55c5172efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 -- 172.21.15.31:0/3636034547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 msgr2=0x55c5172e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ece59c640 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.257+0000 7f3ebf7fe640 1 -- 172.21.15.31:0/3370672652 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3ec40ef570 con 0x7f3ed0102b20 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3ece59c640 1 -- 172.21.15.31:0/3636034547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c516f60210 con 0x55c5172e8840 2024-04-25T00:34:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3eced9d640 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3ece59c640 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172ed340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b7d0 tx=0x7f3ec0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed5994bd733fb17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3eced9d640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3eb80a2c30 0x7f3eb80a50f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55c5172e90a0 tx=0x7f3ec8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3ebcff9640 1 -- 172.21.15.31:0/3636034547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0003dd0 con 0x55c5172e8840 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3ebcff9640 1 -- 172.21.15.31:0/3636034547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec0015070 con 0x55c5172e8840 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.258+0000 7f3ebcff9640 1 -- 172.21.15.31:0/3636034547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00116d0 con 0x55c5172e8840 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- 172.21.15.31:0/3636034547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 msgr2=0x55c5172ed340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172ed340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b7d0 tx=0x7f3ec0004610 comp rx=0 tx=0).stop 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- 172.21.15.31:0/3636034547 shutdown_connections 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172ed340 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/3636034547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- 172.21.15.31:0/3636034547 >> 172.21.15.31:0/3636034547 conn(0x55c5172e34c0 msgr2=0x55c5172e38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- 172.21.15.31:0/3636034547 shutdown_connections 2024-04-25T00:34:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- 172.21.15.31:0/3636034547 wait complete. 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- start start 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 Processor -- start 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c5172f0960 con 0x55c5172e8840 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c517207b50 con 0x55c5172e7e50 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.259+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c5172f0ae0 con 0x55c5172ed9b0 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ed77cf740 10 client.0 _ll_register_callbacks cb 0x55c517242ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47840/0 (socket says 172.21.15.31:47840) 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/282159434 learned_addr learned my addr 172.21.15.31:0/282159434 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ece59c640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3eced9d640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 msgr2=0x55c5172f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/282159434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 msgr2=0x55c5172f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/282159434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51702c620 con 0x55c5172e8840 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f27e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40ed040 tx=0x7f3ec408add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f602dc6e93bcb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/282159434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4049040 con 0x55c5172e8840 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/282159434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3ec40b84c0 con 0x55c5172e8840 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec0015070 con 0x55c5172e8840 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/282159434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ec40bb990 con 0x55c5172e8840 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ed77cf740 10 client.15657 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.260+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c5172fcb90 con 0x55c5172e8840 2024-04-25T00:34:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/282159434 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec0015070 con 0x55c5172e8840 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/282159434 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec40bc0b0 con 0x55c5172e8840 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ed77cf740 10 client.15657 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ed77cf740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ed77cf740 20 client.15657 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c5172fe2a0 0x55c517300730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ed77cf740 4 client.15657 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5172591f0 con 0x55c5172fe2a0 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.261+0000 7f3ece59c640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c5172fe2a0 0x55c517300730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.262+0000 7f3ece59c640 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c5172fe2a0 0x55c517300730 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93cb22f4ab9d9b51 server_cookie=f6b54a2e1560ca7e in_seq=0 out_seq=0 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.262+0000 7f3eb5ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.262+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/282159434 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5172591f0 con 0x55c5172fe2a0 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.262+0000 7f3eb5ffb640 10 client.15657 handle_command_reply: tid=0 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.262+0000 7f3ed77cf740 1 client.15657 shutdown 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 msgr2=0x55c5172f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f27e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40ed040 tx=0x7f3ec408add0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 shutdown_connections 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c5172fe2a0 0x55c517300730 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172ed9b0 0x55c5172f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3eb5ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f27e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/282159434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172e7e50 0x55c5172f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 >> 172.21.15.31:0/282159434 conn(0x55c5172e34c0 msgr2=0x55c5172e25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 shutdown_connections 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 1 -- 172.21.15.31:0/282159434 wait complete. 2024-04-25T00:34:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.263+0000 7f3ed77cf740 20 client.15657 trim_cache size 0 max 16384 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 Processor -- start 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 -- start start 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01aae10 con 0x55c5172e8840 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01ac950 con 0x55c5172ed9b0 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01ab4e0 con 0x55c5172e7e50 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47850/0 (socket says 172.21.15.31:47850) 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/506778671 learned_addr learned my addr 172.21.15.31:0/506778671 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.278+0000 7f3eced9d640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ece59c640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/506778671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 msgr2=0x55c5172e38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172e38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/506778671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 msgr2=0x55c5172f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/506778671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5172e3060 con 0x55c5172e8840 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3eced9d640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ece59c640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f22a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40b68f0 tx=0x7f3ec40ba360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3d5d4d6e7d0931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3eb7fff640 1 -- 172.21.15.31:0/506778671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4049070 con 0x55c5172e8840 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3eb7fff640 1 -- 172.21.15.31:0/506778671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec4003e20 con 0x55c5172e8840 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3eb7fff640 1 -- 172.21.15.31:0/506778671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4041e90 con 0x55c5172e8840 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ed77cf740 1 -- 172.21.15.31:0/506778671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 msgr2=0x55c5172f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.279+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f22a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40b68f0 tx=0x7f3ec40ba360 comp rx=0 tx=0).stop 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- 172.21.15.31:0/506778671 shutdown_connections 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f22a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/506778671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- 172.21.15.31:0/506778671 >> 172.21.15.31:0/506778671 conn(0x55c5172e34c0 msgr2=0x55c5172de2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- 172.21.15.31:0/506778671 shutdown_connections 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- 172.21.15.31:0/506778671 wait complete. 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- start start 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 Processor -- start 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01ac280 con 0x55c5172e8840 2024-04-25T00:34:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01abbb0 con 0x55c5172ed9b0 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01a8df0 con 0x55c5172e7e50 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ed77cf740 10 client.0 _ll_register_callbacks cb 0x7f3ec4005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ecdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47858/0 (socket says 172.21.15.31:47858) 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/963992123 learned_addr learned my addr 172.21.15.31:0/963992123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.280+0000 7f3eced9d640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ece59c640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 msgr2=0x55c5172f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/963992123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 msgr2=0x55c5172f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ecdd9b640 1 -- 172.21.15.31:0/963992123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5172f7b30 con 0x55c5172e8840 2024-04-25T00:34:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eced9d640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ecdd9b640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f6530 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098d0 tx=0x7f3ec40b8160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01e1f2832aa3196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/963992123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4049070 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/963992123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3ec40ef320 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c517253540 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/963992123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ec408b040 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ed77cf740 10 client.15663 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c5172f7b30 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/963992123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c517253540 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/963992123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec403bae0 con 0x55c5172e8840 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ed77cf740 10 client.15663 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ed77cf740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.281+0000 7f3ed77cf740 20 client.15663 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.282+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c517364990 0x55c517366d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.282+0000 7f3ed77cf740 4 client.15663 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.282+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5171d18e0 con 0x55c517364990 2024-04-25T00:34:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.282+0000 7f3ece59c640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c517364990 0x55c517366d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.282+0000 7f3ece59c640 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c517364990 0x55c517366d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35c5c22fbdc4678 server_cookie=c2c0fb9f0c8e7138 in_seq=0 out_seq=0 2024-04-25T00:34:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.282+0000 7f3eb5ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3eb5ffb640 1 -- 172.21.15.31:0/963992123 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5171d18e0 con 0x55c517364990 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3eb5ffb640 10 client.15663 handle_command_reply: tid=0 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 client.15663 shutdown 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 msgr2=0x55c5172f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f6530 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098d0 tx=0x7f3ec40b8160 comp rx=0 tx=0).stop 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 shutdown_connections 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c517364990 0x55c517366d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5172ed9b0 0x55c5172f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c5172e8840 0x55c5172f6530 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 --2- 172.21.15.31:0/963992123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5172e7e50 0x55c5172f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3eb5ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 >> 172.21.15.31:0/963992123 conn(0x55c5172e34c0 msgr2=0x55c5172e38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 shutdown_connections 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 1 -- 172.21.15.31:0/963992123 wait complete. 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.297+0000 7f3ed77cf740 20 client.15663 trim_cache size 0 max 16384 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:34:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3eb80a2c30 msgr2=0x7f3eb80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3eb80a2c30 0x7f3eb80a50f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55c5172e90a0 tx=0x7f3ec8009500 comp rx=0 tx=0).stop 2024-04-25T00:34:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 msgr2=0x7f3ed0100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0100ad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40314e0 tx=0x7f3ec4031a30 comp rx=0 tx=0).stop 2024-04-25T00:34:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 shutdown_connections 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed01034e0 0x7f3ed0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3eb80a2c30 0x7f3eb80a50f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed0102b20 0x7f3ed0100ad0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 --2- 172.21.15.31:0/3370672652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ed00ffab0 0x7f3ed0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 >> 172.21.15.31:0/3370672652 conn(0x7f3ed00fb3c0 msgr2=0x7f3ed0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 shutdown_connections 2024-04-25T00:34:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:00.298+0000 7f3ed49ed640 1 -- 172.21.15.31:0/3370672652 wait complete. 2024-04-25T00:34:01.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:01 smithi031 ceph-mon[29167]: pgmap v464: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:01.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:01 smithi169 ceph-mon[33112]: pgmap v464: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:01.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:01 smithi040 ceph-mon[32945]: pgmap v464: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:03.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:03 smithi031 ceph-mon[29167]: pgmap v465: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:34:03.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:03 smithi169 ceph-mon[33112]: pgmap v465: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:34:03.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:03 smithi040 ceph-mon[32945]: pgmap v465: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:34:05.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:05 smithi031 ceph-mon[29167]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:34:05.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:05 smithi169 ceph-mon[33112]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:34:05.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:05 smithi040 ceph-mon[32945]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:34:06.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:06 smithi031 ceph-mon[29167]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:06.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:06 smithi169 ceph-mon[33112]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:06.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:06 smithi040 ceph-mon[32945]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:08.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:08 smithi031 ceph-mon[29167]: pgmap v468: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:34:08.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:08 smithi169 ceph-mon[33112]: pgmap v468: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:34:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:08 smithi040 ceph-mon[32945]: pgmap v468: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:34:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:10 smithi031 ceph-mon[29167]: pgmap v469: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:10 smithi169 ceph-mon[33112]: pgmap v469: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:10 smithi040 ceph-mon[32945]: pgmap v469: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:12 smithi031 ceph-mon[29167]: pgmap v470: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:34:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:12 smithi169 ceph-mon[33112]: pgmap v470: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:34:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:12 smithi040 ceph-mon[32945]: pgmap v470: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:34:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 -- 172.21.15.31:0/2582428916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c102300 msgr2=0x7ff49c108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/2582428916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c102300 0x7ff49c108b90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff494009ca0 tx=0x7ff49402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 -- 172.21.15.31:0/2582428916 shutdown_connections 2024-04-25T00:34:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/2582428916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c102300 0x7ff49c108b90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/2582428916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c101940 0x7ff49c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/2582428916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c100750 0x7ff49c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.402+0000 7ff4a3f42640 1 -- 172.21.15.31:0/2582428916 >> 172.21.15.31:0/2582428916 conn(0x7ff49c0fbcc0 msgr2=0x7ff49c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 -- 172.21.15.31:0/2582428916 shutdown_connections 2024-04-25T00:34:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 -- 172.21.15.31:0/2582428916 wait complete. 2024-04-25T00:34:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 Processor -- start 2024-04-25T00:34:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 -- start start 2024-04-25T00:34:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 0x7ff49c1046e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c101940 0x7ff49c102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c102300 0x7ff49c103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10c210 con 0x7ff49c100750 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10c090 con 0x7ff49c102300 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.403+0000 7ff4a3f42640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10c390 con 0x7ff49c101940 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 0x7ff49c1046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 0x7ff49c1046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43622/0 (socket says 172.21.15.31:43622) 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1783942890 learned_addr learned my addr 172.21.15.31:0/1783942890 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c101940 0x7ff49c102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c102300 0x7ff49c103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1783942890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c101940 msgr2=0x7ff49c102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c101940 0x7ff49c102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1783942890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c102300 msgr2=0x7ff49c103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c102300 0x7ff49c103260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1783942890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c103af0 con 0x7ff49c100750 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c102300 0x7ff49c103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 0x7ff49c1046e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff49800d770 tx=0x7ff49800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff492ffd640 1 -- 172.21.15.31:0/1783942890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498017070 con 0x7ff49c100750 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c1a19d0 con 0x7ff49c100750 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff492ffd640 1 -- 172.21.15.31:0/1783942890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff498009c30 con 0x7ff49c100750 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff492ffd640 1 -- 172.21.15.31:0/1783942890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498003d00 con 0x7ff49c100750 2024-04-25T00:34:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.404+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a1f20 con 0x7ff49c100750 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 Processor -- start 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff492ffd640 1 -- 172.21.15.31:0/1783942890 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff498003ea0 con 0x7ff49c100750 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 -- start start 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d12a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d17b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d1a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db44c6dcb0 con 0x55db44d12660 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db44c165b0 con 0x55db44d181c0 2024-04-25T00:34:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55db44c69c70 con 0x55db44d13050 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a24df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d12a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a24df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d12a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43628/0 (socket says 172.21.15.31:43628) 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.407+0000 7ff4a24df640 1 -- 172.21.15.31:0/1648315268 learned_addr learned my addr 172.21.15.31:0/1648315268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d1a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d17b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff492ffd640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4880a2c30 0x7ff4880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 -- 172.21.15.31:0/1648315268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 msgr2=0x55db44d17b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d17b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 -- 172.21.15.31:0/1648315268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 msgr2=0x55db44d1a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d1a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 -- 172.21.15.31:0/1648315268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db44982210 con 0x55db44d12660 2024-04-25T00:34:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d17b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff492ffd640 1 -- 172.21.15.31:0/1783942890 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff4980cee40 con 0x7ff49c100750 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4880a2c30 0x7ff4880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d12a80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff494005cc0 tx=0x7ff494005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97bd1cf5a2a8f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff487fff640 1 -- 172.21.15.31:0/1648315268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494004040 con 0x55db44d12660 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff487fff640 1 -- 172.21.15.31:0/1648315268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494036070 con 0x55db44d12660 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff487fff640 1 -- 172.21.15.31:0/1648315268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494030bd0 con 0x55db44d12660 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.408+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4880a2c30 0x7ff4880a50f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff49404b000 tx=0x7ff494048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1648315268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 msgr2=0x55db44d12a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d12a80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff494005cc0 tx=0x7ff494005ef0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1648315268 shutdown_connections 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d1a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d17b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1648315268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d12a80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1648315268 >> 172.21.15.31:0/1648315268 conn(0x55db44d054c0 msgr2=0x55db44d058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1648315268 shutdown_connections 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.409+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1648315268 wait complete. 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 -- start start 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 Processor -- start 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d12660 0x55db44d1c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d1cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d23e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db44d1aff0 con 0x55db44d181c0 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db44c29b50 con 0x55db44d12660 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55db44d1b170 con 0x55db44d13050 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a6c53740 10 client.0 _ll_register_callbacks cb 0x55db44c64ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a24df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d23e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a24df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d23e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43640/0 (socket says 172.21.15.31:43640) 2024-04-25T00:34:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.410+0000 7ff4a24df640 1 -- 172.21.15.31:0/3960977756 learned_addr learned my addr 172.21.15.31:0/3960977756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d12660 0x55db44d1c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d1cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a24df640 1 -- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 msgr2=0x55db44d1cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a24df640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d1cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a24df640 1 -- 172.21.15.31:0/3960977756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d12660 msgr2=0x55db44d1c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a24df640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d12660 0x55db44d1c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a24df640 1 -- 172.21.15.31:0/3960977756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db44a4e620 con 0x55db44d181c0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d12660 0x55db44d1c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a24df640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d23e20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff4940042a0 tx=0x7ff4940042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b319f5c014409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4857fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4857fa640 1 -- 172.21.15.31:0/3960977756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49403f030 con 0x55db44d181c0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4857fa640 1 -- 172.21.15.31:0/3960977756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff49404d050 con 0x55db44d181c0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff494036070 con 0x55db44d181c0 2024-04-25T00:34:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4857fa640 1 -- 172.21.15.31:0/3960977756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff494031750 con 0x55db44d181c0 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a6c53740 10 client.15675 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55db44d1aff0 con 0x55db44d181c0 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.411+0000 7ff4857fa640 1 -- 172.21.15.31:0/3960977756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494036070 con 0x55db44d181c0 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4857fa640 1 -- 172.21.15.31:0/3960977756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff494064070 con 0x55db44d181c0 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a6c53740 10 client.15675 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a6c53740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a6c53740 20 client.15675 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d28520 0x55db44d2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a6c53740 4 client.15675 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db44c7b1f0 con 0x55db44d28520 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.412+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d28520 0x55db44d2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4857fa640 10 client.15675 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d28520 0x55db44d2a910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fddbfe04f63f35 server_cookie=340bb1a40748fe17 in_seq=0 out_seq=0 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4857fa640 1 -- 172.21.15.31:0/3960977756 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db44c7b1f0 con 0x55db44d28520 2024-04-25T00:34:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4857fa640 10 client.15675 handle_command_reply: tid=0 2024-04-25T00:34:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4a6c53740 1 client.15675 shutdown 2024-04-25T00:34:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 msgr2=0x55db44d23e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.413+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d23e20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff4940042a0 tx=0x7ff4940042d0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 shutdown_connections 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d28520 0x55db44d2a910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d23e20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d1cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/3960977756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d12660 0x55db44d1c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4857fa640 0 client.15675 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 >> 172.21.15.31:0/3960977756 conn(0x55db44d054c0 msgr2=0x55db44d045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 shutdown_connections 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 1 -- 172.21.15.31:0/3960977756 wait complete. 2024-04-25T00:34:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.414+0000 7ff4a6c53740 20 client.15675 trim_cache size 0 max 16384 2024-04-25T00:34:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 Processor -- start 2024-04-25T00:34:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 -- start start 2024-04-25T00:34:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d14980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d23de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1a6980 con 0x55db44d12660 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1a84c0 con 0x55db44d181c0 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1a7050 con 0x55db44d13050 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d14440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43648/0 (socket says 172.21.15.31:43648) 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1041308668 learned_addr learned my addr 172.21.15.31:0/1041308668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d14980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1041308668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 msgr2=0x55db44d14980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d14980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1041308668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 msgr2=0x55db44d23de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d23de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d23de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a1cde640 1 -- 172.21.15.31:0/1041308668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db44d17ea0 con 0x55db44d12660 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.429+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d14980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d14440 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff49809b600 tx=0x7ff4980d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5c22971658a9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4877fe640 1 -- 172.21.15.31:0/1041308668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498020040 con 0x55db44d12660 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4877fe640 1 -- 172.21.15.31:0/1041308668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff49809b670 con 0x55db44d12660 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4877fe640 1 -- 172.21.15.31:0/1041308668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4980a3e50 con 0x55db44d12660 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1041308668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 msgr2=0x55db44d14440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d14440 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff49809b600 tx=0x7ff4980d4040 comp rx=0 tx=0).stop 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1041308668 shutdown_connections 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d181c0 0x55db44d23de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d13050 0x55db44d14980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1041308668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d12660 0x55db44d14440 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1041308668 >> 172.21.15.31:0/1041308668 conn(0x55db44d054c0 msgr2=0x55db44d058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1041308668 shutdown_connections 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.430+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1041308668 wait complete. 2024-04-25T00:34:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 -- start start 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 Processor -- start 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d12660 0x55db44d214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d13050 0x55db44d219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1a7df0 con 0x55db44d181c0 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1a7720 con 0x55db44d13050 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1a49a0 con 0x55db44d12660 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a6c53740 10 client.0 _ll_register_callbacks cb 0x7ff49404a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a24df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a24df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d229f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43660/0 (socket says 172.21.15.31:43660) 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a24df640 1 -- 172.21.15.31:0/1644439250 learned_addr learned my addr 172.21.15.31:0/1644439250 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.431+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d13050 0x55db44d219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d12660 0x55db44d214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a24df640 1 -- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d12660 msgr2=0x55db44d214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d12660 0x55db44d214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a24df640 1 -- 172.21.15.31:0/1644439250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d13050 msgr2=0x55db44d219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d13050 0x55db44d219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a24df640 1 -- 172.21.15.31:0/1644439250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db44d22fe0 con 0x55db44d181c0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d12660 0x55db44d214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a14dd640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d13050 0x55db44d219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a24df640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d229f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff494009ca0 tx=0x7ff4940048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160fab7c65489fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4857fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4857fa640 1 -- 172.21.15.31:0/1644439250 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494031750 con 0x55db44d181c0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4857fa640 1 -- 172.21.15.31:0/1644439250 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff49404d040 con 0x55db44d181c0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff49809b670 con 0x55db44d181c0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4857fa640 1 -- 172.21.15.31:0/1644439250 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff494051690 con 0x55db44d181c0 2024-04-25T00:34:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a6c53740 10 client.15681 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55db44d8ea90 con 0x55db44d181c0 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.432+0000 7ff4857fa640 1 -- 172.21.15.31:0/1644439250 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff49809b670 con 0x55db44d181c0 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4857fa640 1 -- 172.21.15.31:0/1644439250 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff494064040 con 0x55db44d181c0 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a6c53740 10 client.15681 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a6c53740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a6c53740 20 client.15681 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d8f280 0x55db44d916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a6c53740 4 client.15681 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db44c75540 con 0x55db44d8f280 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d8f280 0x55db44d916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4a1cde640 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d8f280 0x55db44d916a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868ccdea7214dad8 server_cookie=fa3c9639d666ed95 in_seq=0 out_seq=0 2024-04-25T00:34:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.433+0000 7ff4857fa640 10 client.15681 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.447+0000 7ff4857fa640 1 -- 172.21.15.31:0/1644439250 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db44c75540 con 0x55db44d8f280 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.447+0000 7ff4857fa640 10 client.15681 handle_command_reply: tid=0 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.447+0000 7ff4a6c53740 1 client.15681 shutdown 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 msgr2=0x55db44d229f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d229f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff494009ca0 tx=0x7ff4940048b0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 shutdown_connections 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55db44d8f280 0x55db44d916a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44d181c0 0x55db44d229f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4857fa640 0 client.15681 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db44d13050 0x55db44d219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 --2- 172.21.15.31:0/1644439250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db44d12660 0x55db44d214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 >> 172.21.15.31:0/1644439250 conn(0x55db44d054c0 msgr2=0x55db44d1c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 shutdown_connections 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 1 -- 172.21.15.31:0/1644439250 wait complete. 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a6c53740 20 client.15681 trim_cache size 0 max 16384 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:34:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:34:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.448+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4880a2c30 msgr2=0x7ff4880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4880a2c30 0x7ff4880a50f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff49404b000 tx=0x7ff494048000 comp rx=0 tx=0).stop 2024-04-25T00:34:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 msgr2=0x7ff49c1046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 0x7ff49c1046e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff49800d770 tx=0x7ff49800dc30 comp rx=0 tx=0).stop 2024-04-25T00:34:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 shutdown_connections 2024-04-25T00:34:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49c102300 0x7ff49c103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c101940 0x7ff49c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4880a2c30 0x7ff4880a50f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 --2- 172.21.15.31:0/1783942890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff49c100750 0x7ff49c1046e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 >> 172.21.15.31:0/1783942890 conn(0x7ff49c0fbcc0 msgr2=0x7ff49c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 shutdown_connections 2024-04-25T00:34:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:13.449+0000 7ff4a3f42640 1 -- 172.21.15.31:0/1783942890 wait complete. 2024-04-25T00:34:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:14 smithi031 ceph-mon[29167]: pgmap v471: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:34:14.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:14 smithi169 ceph-mon[33112]: pgmap v471: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:34:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:14 smithi040 ceph-mon[32945]: pgmap v471: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:34:16.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:16 smithi031 ceph-mon[29167]: pgmap v472: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:34:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:16 smithi169 ceph-mon[33112]: pgmap v472: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:34:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:16 smithi040 ceph-mon[32945]: pgmap v472: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:34:18.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:18 smithi031 ceph-mon[29167]: pgmap v473: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:34:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:18 smithi169 ceph-mon[33112]: pgmap v473: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:34:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:18 smithi040 ceph-mon[32945]: pgmap v473: 97 pgs: 97 active+clean; 2.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:34:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:20 smithi031 ceph-mon[29167]: pgmap v474: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:34:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:20 smithi169 ceph-mon[33112]: pgmap v474: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:34:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:20 smithi040 ceph-mon[32945]: pgmap v474: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:34:22.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:22 smithi031 ceph-mon[29167]: pgmap v475: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:22 smithi169 ceph-mon[33112]: pgmap v475: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:22 smithi040 ceph-mon[32945]: pgmap v475: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:24 smithi031 ceph-mon[29167]: pgmap v476: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:34:24.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:24 smithi169 ceph-mon[33112]: pgmap v476: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:34:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:24 smithi040 ceph-mon[32945]: pgmap v476: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:34:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 -- 172.21.15.31:0/171970826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58101940 msgr2=0x7f3a58101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 --2- 172.21.15.31:0/171970826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58101940 0x7f3a58101dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009900 tx=0x7f3a4002a410 comp rx=0 tx=0).stop 2024-04-25T00:34:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 -- 172.21.15.31:0/171970826 shutdown_connections 2024-04-25T00:34:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 --2- 172.21.15.31:0/171970826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a58102300 0x7f3a58108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 --2- 172.21.15.31:0/171970826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58101940 0x7f3a58101dc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 --2- 172.21.15.31:0/171970826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a58100750 0x7f3a58100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 -- 172.21.15.31:0/171970826 >> 172.21.15.31:0/171970826 conn(0x7f3a580fbcc0 msgr2=0x7f3a580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.548+0000 7f3a5d963640 1 -- 172.21.15.31:0/171970826 shutdown_connections 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 -- 172.21.15.31:0/171970826 wait complete. 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 Processor -- start 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 -- start start 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a58100750 0x7f3a58197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a58101940 0x7f3a58197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 0x7f3a5819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5810d330 con 0x7f3a58102300 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5810d1b0 con 0x7f3a58100750 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a5d963640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5810d4b0 con 0x7f3a58101940 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 0x7f3a5819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 0x7f3a5819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50986/0 (socket says 172.21.15.31:50986) 2024-04-25T00:34:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a577fe640 1 -- 172.21.15.31:0/1834475168 learned_addr learned my addr 172.21.15.31:0/1834475168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.549+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a58100750 0x7f3a58197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a58101940 0x7f3a58197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a577fe640 1 -- 172.21.15.31:0/1834475168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a58101940 msgr2=0x7f3a58197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a577fe640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a58101940 0x7f3a58197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a577fe640 1 -- 172.21.15.31:0/1834475168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a58100750 msgr2=0x7f3a58197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a577fe640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a58100750 0x7f3a58197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a577fe640 1 -- 172.21.15.31:0/1834475168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5819f520 con 0x7f3a58102300 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a577fe640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 0x7f3a5819eeb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000eaa0 tx=0x7f3a5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a5c961640 1 -- 172.21.15.31:0/1834475168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000cdb0 con 0x7f3a58102300 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a5c961640 1 -- 172.21.15.31:0/1834475168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a50005020 con 0x7f3a58102300 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5819f810 con 0x7f3a58102300 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a5c961640 1 -- 172.21.15.31:0/1834475168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500116d0 con 0x7f3a58102300 2024-04-25T00:34:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.550+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5819fd80 con 0x7f3a58102300 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a5c961640 1 -- 172.21.15.31:0/1834475168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a500051d0 con 0x7f3a58102300 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 Processor -- start 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 -- start start 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561a71134cb0 con 0x561a711d69b0 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561a710dd5b0 con 0x561a711d1840 2024-04-25T00:34:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a60678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a71130c70 con 0x561a711d0e50 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a5c961640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3a440a2cb0 0x7f3a440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711d8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50996/0 (socket says 172.21.15.31:50996) 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a4ffff640 1 -- 172.21.15.31:0/315704424 learned_addr learned my addr 172.21.15.31:0/315704424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.553+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a577fe640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3a440a2cb0 0x7f3a440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a56ffd640 1 -- 172.21.15.31:0/315704424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 msgr2=0x561a711d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a5c961640 1 -- 172.21.15.31:0/1834475168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3a500cf110 con 0x7f3a58102300 2024-04-25T00:34:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a56ffd640 1 -- 172.21.15.31:0/315704424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 msgr2=0x561a711d8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711d8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a56ffd640 1 -- 172.21.15.31:0/315704424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a70e49210 con 0x561a711d1840 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711d6340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800bbc0 tx=0x7f3a480107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ec15bbc760e530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a577fe640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3a440a2cb0 0x7f3a440a5170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3a5009b6f0 tx=0x7f3a5006eab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a4dffb640 1 -- 172.21.15.31:0/315704424 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48004030 con 0x561a711d1840 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a4dffb640 1 -- 172.21.15.31:0/315704424 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a48005020 con 0x561a711d1840 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.554+0000 7f3a4dffb640 1 -- 172.21.15.31:0/315704424 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48004b80 con 0x561a711d1840 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 -- 172.21.15.31:0/315704424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 msgr2=0x561a711d6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711d6340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800bbc0 tx=0x7f3a480107a0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 -- 172.21.15.31:0/315704424 shutdown_connections 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711d8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711d6340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 --2- 172.21.15.31:0/315704424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 -- 172.21.15.31:0/315704424 >> 172.21.15.31:0/315704424 conn(0x561a711cc4c0 msgr2=0x561a711cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 -- 172.21.15.31:0/315704424 shutdown_connections 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 -- 172.21.15.31:0/315704424 wait complete. 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.555+0000 7f3a60678740 1 -- start start 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 Processor -- start 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d0e50 0x561a711db250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 0x561a711db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d69b0 0x561a711e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561a711d9960 con 0x561a711d1840 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561a710f0b50 con 0x561a711d0e50 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a711d9ae0 con 0x561a711d69b0 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a60678740 10 client.0 _ll_register_callbacks cb 0x561a7112bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 0x561a711db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 0x561a711db790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51010/0 (socket says 172.21.15.31:51010) 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 -- 172.21.15.31:0/1033552629 learned_addr learned my addr 172.21.15.31:0/1033552629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d0e50 0x561a711db250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d69b0 0x561a711e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 -- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d69b0 msgr2=0x561a711e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d69b0 0x561a711e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 -- 172.21.15.31:0/1033552629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d0e50 msgr2=0x561a711db250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d0e50 0x561a711db250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 -- 172.21.15.31:0/1033552629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a70f15620 con 0x561a711d1840 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d0e50 0x561a711db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.556+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 0x561a711db790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b620 tx=0x7f3a4800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd667eb772b52ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a36ffd640 1 -- 172.21.15.31:0/1033552629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4802a060 con 0x561a711d1840 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a36ffd640 1 -- 172.21.15.31:0/1033552629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3a48004040 con 0x561a711d1840 2024-04-25T00:34:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a48005020 con 0x561a711d1840 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a36ffd640 1 -- 172.21.15.31:0/1033552629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a48004240 con 0x561a711d1840 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 10 client.15693 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a711e5b40 con 0x561a711d1840 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a36ffd640 1 -- 172.21.15.31:0/1033552629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a48005020 con 0x561a711d1840 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a36ffd640 1 -- 172.21.15.31:0/1033552629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3a480038f0 con 0x561a711d1840 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 20 client.15693 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a711e6f40 0x561a711e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 4 client.15693 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.557+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a711421f0 con 0x561a711e6f40 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.558+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a711e6f40 0x561a711e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.558+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a711e6f40 0x561a711e9330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301c35bfd7a97e26 server_cookie=3679333cc441c1cf in_seq=0 out_seq=0 2024-04-25T00:34:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.558+0000 7f3a36ffd640 10 client.15693 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a36ffd640 1 -- 172.21.15.31:0/1033552629 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a711421f0 con 0x561a711e6f40 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a36ffd640 10 client.15693 handle_command_reply: tid=0 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 client.15693 shutdown 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 msgr2=0x561a711db790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 0x561a711db790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b620 tx=0x7f3a4800bfa0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 shutdown_connections 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a711e6f40 0x561a711e9330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d69b0 0x561a711e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d1840 0x561a711db790 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a36ffd640 0 client.15693 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 --2- 172.21.15.31:0/1033552629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d0e50 0x561a711db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 >> 172.21.15.31:0/1033552629 conn(0x561a711cc4c0 msgr2=0x561a711cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 shutdown_connections 2024-04-25T00:34:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 1 -- 172.21.15.31:0/1033552629 wait complete. 2024-04-25T00:34:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.559+0000 7f3a60678740 20 client.15693 trim_cache size 0 max 16384 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.574+0000 7f3a60678740 1 Processor -- start 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.574+0000 7f3a60678740 1 -- start start 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a60678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a60678740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a60678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a60678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581a8c40 con 0x561a711d69b0 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a60678740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581aa780 con 0x561a711d1840 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a60678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581a9310 con 0x561a711d0e50 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51018/0 (socket says 172.21.15.31:51018) 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 -- 172.21.15.31:0/1042524557 learned_addr learned my addr 172.21.15.31:0/1042524557 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 -- 172.21.15.31:0/1042524557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 msgr2=0x561a711d8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 -- 172.21.15.31:0/1042524557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 msgr2=0x561a711e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711e2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 -- 172.21.15.31:0/1042524557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a711e9960 con 0x561a711d69b0 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711cc8a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3a4002a930 tx=0x7f3a40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462f28d2eb871495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4d7fa640 1 -- 172.21.15.31:0/1042524557 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40051070 con 0x561a711d69b0 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4d7fa640 1 -- 172.21.15.31:0/1042524557 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4004a900 con 0x561a711d69b0 2024-04-25T00:34:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.575+0000 7f3a4d7fa640 1 -- 172.21.15.31:0/1042524557 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40035040 con 0x561a711d69b0 2024-04-25T00:34:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 -- 172.21.15.31:0/1042524557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 msgr2=0x561a711cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711cc8a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3a4002a930 tx=0x7f3a40009260 comp rx=0 tx=0).stop 2024-04-25T00:34:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 -- 172.21.15.31:0/1042524557 shutdown_connections 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d69b0 0x561a711cc8a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d1840 0x561a711e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 --2- 172.21.15.31:0/1042524557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d0e50 0x561a711d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 -- 172.21.15.31:0/1042524557 >> 172.21.15.31:0/1042524557 conn(0x561a711cc4c0 msgr2=0x561a711c72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 -- 172.21.15.31:0/1042524557 shutdown_connections 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 -- 172.21.15.31:0/1042524557 wait complete. 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.576+0000 7f3a60678740 1 -- start start 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 Processor -- start 2024-04-25T00:34:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 0x561a711dfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d1840 0x561a711e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d69b0 0x561a711e1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581aa0b0 con 0x561a711d0e50 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581a99e0 con 0x561a711d69b0 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581a6c20 con 0x561a711d1840 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a60678740 10 client.0 _ll_register_callbacks cb 0x7f3a40032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 0x561a711dfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 0x561a711dfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51026/0 (socket says 172.21.15.31:51026) 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 -- 172.21.15.31:0/2524854481 learned_addr learned my addr 172.21.15.31:0/2524854481 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d1840 0x561a711e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d69b0 0x561a711e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 -- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d1840 msgr2=0x561a711e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d1840 0x561a711e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 -- 172.21.15.31:0/2524854481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d69b0 msgr2=0x561a711e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d69b0 0x561a711e1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 -- 172.21.15.31:0/2524854481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a711e1620 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a4ffff640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 0x561a711dfb50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009630 tx=0x7f3a40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e062cf056cc484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.577+0000 7f3a36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a36ffd640 1 -- 172.21.15.31:0/2524854481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40000e60 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a36ffd640 1 -- 172.21.15.31:0/2524854481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3a40004450 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a4004a900 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a36ffd640 1 -- 172.21.15.31:0/2524854481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a40035040 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 10 client.15699 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a711e2930 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a36ffd640 1 -- 172.21.15.31:0/2524854481 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4004a900 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a36ffd640 1 -- 172.21.15.31:0/2524854481 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3a4004bce0 con 0x561a711d0e50 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 10 client.15699 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 20 client.15699 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a7124d9d0 0x561a7124fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 4 client.15699 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.578+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a7113c540 con 0x561a7124d9d0 2024-04-25T00:34:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.579+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a7124d9d0 0x561a7124fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.579+0000 7f3a36ffd640 10 client.15699 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.579+0000 7f3a56ffd640 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a7124d9d0 0x561a7124fdf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d855dc1c788c69b server_cookie=2775ac3c1f27291e in_seq=0 out_seq=0 2024-04-25T00:34:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a36ffd640 1 -- 172.21.15.31:0/2524854481 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a7113c540 con 0x561a7124d9d0 2024-04-25T00:34:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a36ffd640 10 client.15699 handle_command_reply: tid=0 2024-04-25T00:34:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 client.15699 shutdown 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 msgr2=0x561a711dfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 0x561a711dfb50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009630 tx=0x7f3a40000bc0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 shutdown_connections 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x561a7124d9d0 0x561a7124fdf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a711d69b0 0x561a711e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a711d1840 0x561a711e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 --2- 172.21.15.31:0/2524854481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a711d0e50 0x561a711dfb50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a36ffd640 0 client.15699 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 >> 172.21.15.31:0/2524854481 conn(0x561a711cc4c0 msgr2=0x561a711cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 shutdown_connections 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.594+0000 7f3a60678740 1 -- 172.21.15.31:0/2524854481 wait complete. 2024-04-25T00:34:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a60678740 20 client.15699 trim_cache size 0 max 16384 2024-04-25T00:34:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:34:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:34:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:34:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:34:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3a440a2cb0 msgr2=0x7f3a440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3a440a2cb0 0x7f3a440a5170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3a5009b6f0 tx=0x7f3a5006eab0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 msgr2=0x7f3a5819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 0x7f3a5819eeb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000eaa0 tx=0x7f3a5000ef60 comp rx=0 tx=0).stop 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 shutdown_connections 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3a440a2cb0 0x7f3a440a5170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a58102300 0x7f3a5819eeb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a58101940 0x7f3a58197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 --2- 172.21.15.31:0/1834475168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a58100750 0x7f3a58197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 >> 172.21.15.31:0/1834475168 conn(0x7f3a580fbcc0 msgr2=0x7f3a58104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 shutdown_connections 2024-04-25T00:34:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:26.595+0000 7f3a5d963640 1 -- 172.21.15.31:0/1834475168 wait complete. 2024-04-25T00:34:26.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:26 smithi031 ceph-mon[29167]: pgmap v477: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:26 smithi169 ceph-mon[33112]: pgmap v477: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:26 smithi040 ceph-mon[32945]: pgmap v477: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:28.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:28 smithi031 ceph-mon[29167]: pgmap v478: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:34:28.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:28 smithi169 ceph-mon[33112]: pgmap v478: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:34:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:28 smithi040 ceph-mon[32945]: pgmap v478: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:34:30.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:30 smithi031 ceph-mon[29167]: pgmap v479: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:34:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:30 smithi169 ceph-mon[33112]: pgmap v479: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:34:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:30 smithi040 ceph-mon[32945]: pgmap v479: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:34:32.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:32 smithi031 ceph-mon[29167]: pgmap v480: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:32 smithi169 ceph-mon[33112]: pgmap v480: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:32 smithi040 ceph-mon[32945]: pgmap v480: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:34.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:34 smithi031 ceph-mon[29167]: pgmap v481: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:34:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:34 smithi169 ceph-mon[33112]: pgmap v481: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:34:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:34 smithi040 ceph-mon[32945]: pgmap v481: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:34:36.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:36 smithi031 ceph-mon[29167]: pgmap v482: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:34:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:36 smithi169 ceph-mon[33112]: pgmap v482: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:34:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:36 smithi040 ceph-mon[32945]: pgmap v482: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:34:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:38 smithi031 ceph-mon[29167]: pgmap v483: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:38 smithi169 ceph-mon[33112]: pgmap v483: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:38 smithi040 ceph-mon[32945]: pgmap v483: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 -- 172.21.15.31:0/924656047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34100750 msgr2=0x7f4a34100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 --2- 172.21.15.31:0/924656047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34100750 0x7f4a34100b70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009900 tx=0x7f4a2402a410 comp rx=0 tx=0).stop 2024-04-25T00:34:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 -- 172.21.15.31:0/924656047 shutdown_connections 2024-04-25T00:34:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 --2- 172.21.15.31:0/924656047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34102300 0x7f4a34108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 --2- 172.21.15.31:0/924656047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a34101940 0x7f4a34101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 --2- 172.21.15.31:0/924656047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34100750 0x7f4a34100b70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.694+0000 7f4a394d4640 1 -- 172.21.15.31:0/924656047 >> 172.21.15.31:0/924656047 conn(0x7f4a340fbcc0 msgr2=0x7f4a340fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 -- 172.21.15.31:0/924656047 shutdown_connections 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 -- 172.21.15.31:0/924656047 wait complete. 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 Processor -- start 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 -- start start 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a34100750 0x7f4a34197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34101940 0x7f4a34197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 0x7f4a3419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3410c200 con 0x7f4a34102300 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3410c080 con 0x7f4a34101940 2024-04-25T00:34:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.695+0000 7f4a394d4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3410c380 con 0x7f4a34100750 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 0x7f4a3419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 0x7f4a3419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46656/0 (socket says 172.21.15.31:46656) 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 -- 172.21.15.31:0/3869357831 learned_addr learned my addr 172.21.15.31:0/3869357831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34101940 0x7f4a34197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 -- 172.21.15.31:0/3869357831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a34100750 msgr2=0x7f4a34197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a34100750 0x7f4a34197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a34100750 0x7f4a34197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 -- 172.21.15.31:0/3869357831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34101940 msgr2=0x7f4a34197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34101940 0x7f4a34197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 -- 172.21.15.31:0/3869357831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3419f5b0 con 0x7f4a34102300 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34101940 0x7f4a34197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a337fe640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 0x7f4a3419ef40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00eb10 tx=0x7f4a2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a30ff9640 1 -- 172.21.15.31:0/3869357831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c004b20 con 0x7f4a34102300 2024-04-25T00:34:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3419f8a0 con 0x7f4a34102300 2024-04-25T00:34:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a30ff9640 1 -- 172.21.15.31:0/3869357831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c004cc0 con 0x7f4a34102300 2024-04-25T00:34:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a30ff9640 1 -- 172.21.15.31:0/3869357831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c010690 con 0x7f4a34102300 2024-04-25T00:34:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.696+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3419fda0 con 0x7f4a34102300 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.699+0000 7f4a30ff9640 1 -- 172.21.15.31:0/3869357831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4a2c005020 con 0x7f4a34102300 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.699+0000 7f4a3c1e9740 1 Processor -- start 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.699+0000 7f4a3c1e9740 1 -- start start 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a30ff9640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a200a2c30 0x7f4a200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dac9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dace750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dacedc0 0x56475dad1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56475da28cb0 con 0x56475dac9c50 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56475d9d15b0 con 0x56475dacedc0 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56475da24c70 con 0x56475dac9260 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a200a2c30 0x7f4a200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a30ff9640 1 -- 172.21.15.31:0/3869357831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a2c0cf770 con 0x7f4a34102300 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dace750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a32ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dac9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a32ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dac9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:51498/0 (socket says 172.21.15.31:51498) 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dace750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46668/0 (socket says 172.21.15.31:46668) 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a32ffd640 1 -- 172.21.15.31:0/2539519670 learned_addr learned my addr 172.21.15.31:0/2539519670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dacedc0 0x56475dad1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a337fe640 1 -- 172.21.15.31:0/2539519670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 msgr2=0x56475dac9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dac9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a337fe640 1 -- 172.21.15.31:0/2539519670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 msgr2=0x56475dace750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dace750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a337fe640 1 -- 172.21.15.31:0/2539519670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56475d73d210 con 0x56475dacedc0 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dac9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.700+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dace750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a200a2c30 0x7f4a200a50f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0058c0 tx=0x7f4a1c00cc00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dacedc0 0x56475dad1250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0cd290 tx=0x7f4a2c09be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685cbc7e9f44eb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a297fa640 1 -- 172.21.15.31:0/2539519670 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0662e0 con 0x56475dacedc0 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a297fa640 1 -- 172.21.15.31:0/2539519670 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c09c6a0 con 0x56475dacedc0 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a297fa640 1 -- 172.21.15.31:0/2539519670 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0a4c10 con 0x56475dacedc0 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2539519670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dacedc0 msgr2=0x56475dad1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.701+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dacedc0 0x56475dad1250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0cd290 tx=0x7f4a2c09be30 comp rx=0 tx=0).stop 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2539519670 shutdown_connections 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dacedc0 0x56475dad1250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dace750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2539519670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dac9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2539519670 >> 172.21.15.31:0/2539519670 conn(0x56475dac04c0 msgr2=0x56475dac08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2539519670 shutdown_connections 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2539519670 wait complete. 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- start start 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 Processor -- start 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9260 0x56475dad3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dad3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475dadacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56475dad1d70 con 0x56475dac9c50 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56475d9e4b50 con 0x56475dac9260 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56475dad1ef0 con 0x56475dacedc0 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.702+0000 7f4a3c1e9740 10 client.0 _ll_register_callbacks cb 0x56475da1fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dad3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a32ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9260 0x56475dad3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dad3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46670/0 (socket says 172.21.15.31:46670) 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 -- 172.21.15.31:0/4208560754 learned_addr learned my addr 172.21.15.31:0/4208560754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a337fe640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475dadacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 -- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 msgr2=0x56475dadacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475dadacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 -- 172.21.15.31:0/4208560754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9260 msgr2=0x56475dad3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9260 0x56475dad3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 -- 172.21.15.31:0/4208560754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475d809620 con 0x56475dac9c50 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9260 0x56475dad3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dad3c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c011be0 tx=0x7f4a1c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653e2d231513a536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/4208560754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c026070 con 0x56475dac9c50 2024-04-25T00:34:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/4208560754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a1c005020 con 0x56475dac9c50 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c09c6a0 con 0x56475dac9c50 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.703+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/4208560754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a1c031050 con 0x56475dac9c50 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 10 client.15711 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56475dade0a0 con 0x56475dac9c50 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/4208560754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c09c6a0 con 0x56475dac9c50 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/4208560754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a1c004cd0 con 0x56475dac9c50 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 10 client.15711 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 20 client.15711 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475dadf740 0x56475dae1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 4 client.15711 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56475da361f0 con 0x56475dadf740 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.704+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475dadf740 0x56475dae1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.705+0000 7f4a1a7fc640 10 client.15711 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.705+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475dadf740 0x56475dae1c00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e238b7566b061d server_cookie=9fe3dc6351a70ed0 in_seq=0 out_seq=0 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/4208560754 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56475da361f0 con 0x56475dadf740 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a1a7fc640 10 client.15711 handle_command_reply: tid=0 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 client.15711 shutdown 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 msgr2=0x56475dad3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dad3c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c011be0 tx=0x7f4a1c005850 comp rx=0 tx=0).stop 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 shutdown_connections 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475dadf740 0x56475dae1c00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475dadacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9c50 0x56475dad3c80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/4208560754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9260 0x56475dad3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 >> 172.21.15.31:0/4208560754 conn(0x56475dac04c0 msgr2=0x56475dabf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a1a7fc640 0 client.15711 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 shutdown_connections 2024-04-25T00:34:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/4208560754 wait complete. 2024-04-25T00:34:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.706+0000 7f4a3c1e9740 20 client.15711 trim_cache size 0 max 16384 2024-04-25T00:34:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 Processor -- start 2024-04-25T00:34:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 -- start start 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dabed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dadac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 0x56475dabf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341a6a10 con 0x56475dacedc0 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341a8550 con 0x56475dac9c50 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.721+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341a70e0 con 0x56475dac9260 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 0x56475dabf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 0x56475dabf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46676/0 (socket says 172.21.15.31:46676) 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 -- 172.21.15.31:0/2511400829 learned_addr learned my addr 172.21.15.31:0/2511400829 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dabed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dadac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 -- 172.21.15.31:0/2511400829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 msgr2=0x56475dabed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dabed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 -- 172.21.15.31:0/2511400829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 msgr2=0x56475dadac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dadac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 -- 172.21.15.31:0/2511400829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56475daca030 con 0x56475dacedc0 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 0x56475dabf270 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c098970 tx=0x7f4a2c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38dd720bc197c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a28ff9640 1 -- 172.21.15.31:0/2511400829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c09c6a0 con 0x56475dacedc0 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a28ff9640 1 -- 172.21.15.31:0/2511400829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c09bfa0 con 0x56475dacedc0 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.722+0000 7f4a28ff9640 1 -- 172.21.15.31:0/2511400829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c098610 con 0x56475dacedc0 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2511400829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 msgr2=0x56475dabf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 0x56475dabf270 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c098970 tx=0x7f4a2c0057a0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2511400829 shutdown_connections 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dacedc0 0x56475dabf270 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dadac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2511400829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dac9260 0x56475dabed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2511400829 >> 172.21.15.31:0/2511400829 conn(0x56475dac04c0 msgr2=0x56475dae1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2511400829 shutdown_connections 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2511400829 wait complete. 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 -- start start 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.723+0000 7f4a3c1e9740 1 Processor -- start 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 0x56475dad7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dad81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475db43d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341a7e80 con 0x56475dac9260 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341a77b0 con 0x56475dac9c50 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341a49f0 con 0x56475dacedc0 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 10 client.0 _ll_register_callbacks cb 0x7f4a1c00c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 0x56475dad7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 0x56475dad7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46684/0 (socket says 172.21.15.31:46684) 2024-04-25T00:34:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 -- 172.21.15.31:0/2198565025 learned_addr learned my addr 172.21.15.31:0/2198565025 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a337fe640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475db43d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dad81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 -- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 msgr2=0x56475db43d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475db43d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 -- 172.21.15.31:0/2198565025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 msgr2=0x56475dad81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dad81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 -- 172.21.15.31:0/2198565025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475db44360 con 0x56475dac9260 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a32ffd640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 0x56475dad7ca0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a24005940 tx=0x7f4a240095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150456604b74b09b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/2198565025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24035040 con 0x56475dac9260 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/2198565025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a24044d50 con 0x56475dac9260 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.724+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56475da30540 con 0x56475dac9260 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/2198565025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a24031e50 con 0x56475dac9260 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a3c1e9740 10 client.15717 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56475db456e0 con 0x56475dac9260 2024-04-25T00:34:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/2198565025 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56475da30540 con 0x56475dac9260 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/2198565025 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a24055070 con 0x56475dac9260 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a3c1e9740 10 client.15717 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a3c1e9740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.725+0000 7f4a3c1e9740 20 client.15717 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.726+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475db460c0 0x56475db484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.726+0000 7f4a3c1e9740 4 client.15717 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.726+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56475d9ae8e0 con 0x56475db460c0 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.726+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475db460c0 0x56475db484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.726+0000 7f4a1a7fc640 10 client.15717 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.726+0000 7f4a2bfff640 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475db460c0 0x56475db484e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ee530cc3661a67 server_cookie=c6ac5c45210d5739 in_seq=0 out_seq=0 2024-04-25T00:34:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.741+0000 7f4a1a7fc640 1 -- 172.21.15.31:0/2198565025 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56475d9ae8e0 con 0x56475db460c0 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.741+0000 7f4a1a7fc640 10 client.15717 handle_command_reply: tid=0 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.741+0000 7f4a3c1e9740 1 client.15717 shutdown 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 msgr2=0x56475dad7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 0x56475dad7ca0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a24005940 tx=0x7f4a240095d0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 shutdown_connections 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56475db460c0 0x56475db484e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56475dacedc0 0x56475db43d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56475dac9c50 0x56475dad81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 --2- 172.21.15.31:0/2198565025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56475dac9260 0x56475dad7ca0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a1a7fc640 0 client.15717 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 >> 172.21.15.31:0/2198565025 conn(0x56475dac04c0 msgr2=0x56475dac3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 shutdown_connections 2024-04-25T00:34:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 1 -- 172.21.15.31:0/2198565025 wait complete. 2024-04-25T00:34:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a3c1e9740 20 client.15717 trim_cache size 0 max 16384 2024-04-25T00:34:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:34:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:34:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:34:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a200a2c30 msgr2=0x7f4a200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a200a2c30 0x7f4a200a50f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0058c0 tx=0x7f4a1c00cc00 comp rx=0 tx=0).stop 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 msgr2=0x7f4a3419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 0x7f4a3419ef40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00eb10 tx=0x7f4a2c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 shutdown_connections 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a200a2c30 0x7f4a200a50f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a34102300 0x7f4a3419ef40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a34101940 0x7f4a34197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 --2- 172.21.15.31:0/3869357831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a34100750 0x7f4a34197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.742+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 >> 172.21.15.31:0/3869357831 conn(0x7f4a340fbcc0 msgr2=0x7f4a340fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.743+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 shutdown_connections 2024-04-25T00:34:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:39.743+0000 7f4a394d4640 1 -- 172.21.15.31:0/3869357831 wait complete. 2024-04-25T00:34:40.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:40 smithi031 ceph-mon[29167]: pgmap v484: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:34:40.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:40 smithi169 ceph-mon[33112]: pgmap v484: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:34:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:40 smithi040 ceph-mon[32945]: pgmap v484: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:34:42.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:42 smithi031 ceph-mon[29167]: pgmap v485: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:34:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:42 smithi169 ceph-mon[33112]: pgmap v485: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:34:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:42 smithi040 ceph-mon[32945]: pgmap v485: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:34:44.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:44 smithi031 ceph-mon[29167]: pgmap v486: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:44 smithi169 ceph-mon[33112]: pgmap v486: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:44 smithi040 ceph-mon[32945]: pgmap v486: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:34:46.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:46 smithi031 ceph-mon[29167]: pgmap v487: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:34:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:46 smithi169 ceph-mon[33112]: pgmap v487: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:34:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:46 smithi040 ceph-mon[32945]: pgmap v487: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:34:48.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:48 smithi031 ceph-mon[29167]: pgmap v488: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:48.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:48 smithi169 ceph-mon[33112]: pgmap v488: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:48.898 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:48 smithi040 ceph-mon[32945]: pgmap v488: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:34:49.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:49 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:34:49.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:49 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:34:50.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:49 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:34:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:50 smithi169 ceph-mon[33112]: pgmap v489: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-04-25T00:34:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:34:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:34:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:34:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:50 smithi040 ceph-mon[32945]: pgmap v489: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-04-25T00:34:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:34:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:34:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:34:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:50 smithi031 ceph-mon[29167]: pgmap v489: 97 pgs: 97 active+clean; 2.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-04-25T00:34:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:34:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:34:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/3589523797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b441006a0 msgr2=0x7f5b44100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/3589523797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b441006a0 0x7f5b44100ac0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c02a410 comp rx=0 tx=0).stop 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/3589523797 shutdown_connections 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/3589523797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b44102250 0x7f5b44108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/3589523797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b44101890 0x7f5b44101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/3589523797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b441006a0 0x7f5b44100ac0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/3589523797 >> 172.21.15.31:0/3589523797 conn(0x7f5b440fbc10 msgr2=0x7f5b440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/3589523797 shutdown_connections 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/3589523797 wait complete. 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.842+0000 7f5b4b0bd640 1 Processor -- start 2024-04-25T00:34:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 -- start start 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b441006a0 0x7f5b441979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b44101890 0x7f5b44197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 0x7f5b4419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d2f0 con 0x7f5b44102250 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d170 con 0x7f5b44101890 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4b0bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d470 con 0x7f5b441006a0 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 0x7f5b4419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 0x7f5b4419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50344/0 (socket says 172.21.15.31:50344) 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 -- 172.21.15.31:0/4063271166 learned_addr learned my addr 172.21.15.31:0/4063271166 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b48e59640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b441006a0 0x7f5b441979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b43fff640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b44101890 0x7f5b44197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 -- 172.21.15.31:0/4063271166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b441006a0 msgr2=0x7f5b441979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b441006a0 0x7f5b441979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 -- 172.21.15.31:0/4063271166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b44101890 msgr2=0x7f5b44197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b44101890 0x7f5b44197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 -- 172.21.15.31:0/4063271166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4419f570 con 0x7f5b44102250 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.843+0000 7f5b4965a640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 0x7f5b4419ef00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d750 tx=0x7f5b3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.844+0000 7f5b41ffb640 1 -- 172.21.15.31:0/4063271166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c017070 con 0x7f5b44102250 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.844+0000 7f5b41ffb640 1 -- 172.21.15.31:0/4063271166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3c0054d0 con 0x7f5b44102250 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.844+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4419f860 con 0x7f5b44102250 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.844+0000 7f5b41ffb640 1 -- 172.21.15.31:0/4063271166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c004c60 con 0x7f5b44102250 2024-04-25T00:34:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.844+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4419fe00 con 0x7f5b44102250 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.846+0000 7f5b41ffb640 1 -- 172.21.15.31:0/4063271166 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5b3c004e00 con 0x7f5b44102250 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 Processor -- start 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 -- start start 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b41ffb640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5b300a2cb0 0x7f5b300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f63639e50 0x564f6363a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6363f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b43fff640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5b300a2cb0 0x7f5b300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363f9b0 0x564f63641e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564f6359dcb0 con 0x564f6363a840 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564f635465b0 con 0x564f6363f9b0 2024-04-25T00:34:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564f63599c70 con 0x564f63639e50 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b41ffb640 1 -- 172.21.15.31:0/4063271166 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f5b3c0ce910 con 0x7f5b44102250 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6363f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6363f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50348/0 (socket says 172.21.15.31:50348) 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b43fff640 1 -- 172.21.15.31:0/180700011 learned_addr learned my addr 172.21.15.31:0/180700011 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b48e59640 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f63639e50 0x564f6363a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b4965a640 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363f9b0 0x564f63641e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b48e59640 1 -- 172.21.15.31:0/180700011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363f9b0 msgr2=0x564f63641e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b48e59640 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363f9b0 0x564f63641e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b48e59640 1 -- 172.21.15.31:0/180700011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 msgr2=0x564f6363f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b48e59640 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6363f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b48e59640 1 -- 172.21.15.31:0/180700011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f632b2210 con 0x564f63639e50 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.847+0000 7f5b43fff640 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6363f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.848+0000 7f5b43fff640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5b300a2cb0 0x7f5b300a5170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009ad0 tx=0x7f5b34003970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.848+0000 7f5b48e59640 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f63639e50 0x564f6363a270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009230 tx=0x7f5b2c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce36cfd22833ecb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.848+0000 7f5b3affd640 1 -- 172.21.15.31:0/180700011 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c035030 con 0x564f63639e50 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.848+0000 7f5b3affd640 1 -- 172.21.15.31:0/180700011 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c040070 con 0x564f63639e50 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.848+0000 7f5b3affd640 1 -- 172.21.15.31:0/180700011 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c003990 con 0x564f63639e50 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/180700011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f63639e50 msgr2=0x564f6363a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f63639e50 0x564f6363a270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009230 tx=0x7f5b2c005c00 comp rx=0 tx=0).stop 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/180700011 shutdown_connections 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363f9b0 0x564f63641e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6363f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/180700011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f63639e50 0x564f6363a270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/180700011 >> 172.21.15.31:0/180700011 conn(0x564f636354c0 msgr2=0x564f636358c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/180700011 shutdown_connections 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/180700011 wait complete. 2024-04-25T00:34:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 -- start start 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.849+0000 7f5b4ddd3740 1 Processor -- start 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63648720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f63648c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564f63642900 con 0x564f6363a840 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564f63559b50 con 0x564f63639e50 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564f63642a80 con 0x564f6363f9b0 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4ddd3740 10 client.0 _ll_register_callbacks cb 0x564f63594ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f63648c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b48e59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63648720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f63648c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50358/0 (socket says 172.21.15.31:50358) 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 -- 172.21.15.31:0/3122068719 learned_addr learned my addr 172.21.15.31:0/3122068719 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b4965a640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 -- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 msgr2=0x564f6364fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 -- 172.21.15.31:0/3122068719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 msgr2=0x564f63648720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63648720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 -- 172.21.15.31:0/3122068719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6337e620 con 0x564f6363a840 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b48e59640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63648720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b43fff640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f63648c60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009aa0 tx=0x7f5b34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4d47db5a1ed5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.850+0000 7f5b23fff640 1 -- 172.21.15.31:0/3122068719 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34026070 con 0x564f6363a840 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2c040070 con 0x564f6363a840 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b23fff640 1 -- 172.21.15.31:0/3122068719 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f5b34013c80 con 0x564f6363a840 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b23fff640 1 -- 172.21.15.31:0/3122068719 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b34031050 con 0x564f6363a840 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 10 client.15729 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f63653080 con 0x564f6363a840 2024-04-25T00:34:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b23fff640 1 -- 172.21.15.31:0/3122068719 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c040070 con 0x564f6363a840 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b23fff640 1 -- 172.21.15.31:0/3122068719 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5b3401e040 con 0x564f6363a840 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 10 client.15729 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 20 client.15729 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f63654770 0x564f63656c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 4 client.15729 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f635ab1f0 con 0x564f63654770 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.851+0000 7f5b48e59640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f63654770 0x564f63656c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.852+0000 7f5b48e59640 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f63654770 0x564f63656c30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3901981ecb4e93bc server_cookie=15827f6afeeceda5 in_seq=0 out_seq=0 2024-04-25T00:34:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.852+0000 7f5b23fff640 10 client.15729 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b23fff640 1 -- 172.21.15.31:0/3122068719 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f635ab1f0 con 0x564f63654770 2024-04-25T00:34:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b23fff640 10 client.15729 handle_command_reply: tid=0 2024-04-25T00:34:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 client.15729 shutdown 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 msgr2=0x564f63648c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f63648c60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009aa0 tx=0x7f5b34004ab0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 shutdown_connections 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f63654770 0x564f63656c30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f63648c60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b23fff640 0 client.15729 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/3122068719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63648720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 >> 172.21.15.31:0/3122068719 conn(0x564f636354c0 msgr2=0x564f636345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 shutdown_connections 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/3122068719 wait complete. 2024-04-25T00:34:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.853+0000 7f5b4ddd3740 20 client.15729 trim_cache size 0 max 16384 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 Processor -- start 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 -- start start 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63638470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6364fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f636389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a69b0 con 0x564f6363a840 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a84f0 con 0x564f63639e50 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.868+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a7080 con 0x564f6363f9b0 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6364fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6364fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50366/0 (socket says 172.21.15.31:50366) 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 -- 172.21.15.31:0/671321358 learned_addr learned my addr 172.21.15.31:0/671321358 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 -- 172.21.15.31:0/671321358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 msgr2=0x564f636389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b4965a640 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f636389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b48e59640 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63638470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f636389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 -- 172.21.15.31:0/671321358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 msgr2=0x564f63638470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63638470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 -- 172.21.15.31:0/671321358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6363ac20 con 0x564f6363a840 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b4965a640 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f636389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b43fff640 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6364fc60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b34020b40 tx=0x7f5b34003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c119e73b92d3a043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b3a7fc640 1 -- 172.21.15.31:0/671321358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34026070 con 0x564f6363a840 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b3a7fc640 1 -- 172.21.15.31:0/671321358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b340040c0 con 0x564f6363a840 2024-04-25T00:34:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.869+0000 7f5b3a7fc640 1 -- 172.21.15.31:0/671321358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34035690 con 0x564f6363a840 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/671321358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 msgr2=0x564f6364fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6364fc60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b34020b40 tx=0x7f5b34003f90 comp rx=0 tx=0).stop 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/671321358 shutdown_connections 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f636389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f6363a840 0x564f6364fc60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/671321358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f63639e50 0x564f63638470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/671321358 >> 172.21.15.31:0/671321358 conn(0x564f636354c0 msgr2=0x564f63646d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/671321358 shutdown_connections 2024-04-25T00:34:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/671321358 wait complete. 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 -- start start 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.870+0000 7f5b4ddd3740 1 Processor -- start 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 0x564f6364cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363a840 0x564f6364d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a7e20 con 0x564f63639e50 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a7750 con 0x564f6363a840 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a4990 con 0x564f6363f9b0 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4ddd3740 10 client.0 _ll_register_callbacks cb 0x7f5b34004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 0x564f6364cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 0x564f6364cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50368/0 (socket says 172.21.15.31:50368) 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 -- 172.21.15.31:0/1066841107 learned_addr learned my addr 172.21.15.31:0/1066841107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b4965a640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b43fff640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363a840 0x564f6364d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 -- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 msgr2=0x564f6364e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 -- 172.21.15.31:0/1066841107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363a840 msgr2=0x564f6364d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363a840 0x564f6364d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 -- 172.21.15.31:0/1066841107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6364e8d0 con 0x564f63639e50 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b43fff640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363a840 0x564f6364d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b48e59640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 0x564f6364cd90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a3c901d134dae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.871+0000 7f5b23fff640 1 -- 172.21.15.31:0/1066841107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c057060 con 0x564f63639e50 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b23fff640 1 -- 172.21.15.31:0/1066841107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f5b2c004240 con 0x564f63639e50 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b340040c0 con 0x564f63639e50 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b23fff640 1 -- 172.21.15.31:0/1066841107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c035040 con 0x564f63639e50 2024-04-25T00:34:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 10 client.15735 fetch_fsmap learned FSMap version 26 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f6364fb80 con 0x564f63639e50 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b23fff640 1 -- 172.21.15.31:0/1066841107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b340040c0 con 0x564f63639e50 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b23fff640 1 -- 172.21.15.31:0/1066841107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5b2c003680 con 0x564f63639e50 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 20 client.15735 populate_metadata read hostname 'smithi031' 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f636badb0 0x564f636bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 4 client.15735 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.872+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f635a5540 con 0x564f636badb0 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.873+0000 7f5b43fff640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f636badb0 0x564f636bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.873+0000 7f5b43fff640 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f636badb0 0x564f636bd1a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675860e599232517 server_cookie=5f53794b45341ab4 in_seq=0 out_seq=0 2024-04-25T00:34:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.873+0000 7f5b23fff640 10 client.15735 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b23fff640 1 -- 172.21.15.31:0/1066841107 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f635a5540 con 0x564f636badb0 2024-04-25T00:34:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b23fff640 10 client.15735 handle_command_reply: tid=0 2024-04-25T00:34:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 client.15735 shutdown 2024-04-25T00:34:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 msgr2=0x564f6364cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 0x564f6364cd90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 shutdown_connections 2024-04-25T00:34:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564f636badb0 0x564f636bd1a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f6363f9b0 0x564f6364e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f6363a840 0x564f6364d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 --2- 172.21.15.31:0/1066841107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f63639e50 0x564f6364cd90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b23fff640 0 client.15735 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 >> 172.21.15.31:0/1066841107 conn(0x564f636354c0 msgr2=0x564f63646950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 shutdown_connections 2024-04-25T00:34:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 1 -- 172.21.15.31:0/1066841107 wait complete. 2024-04-25T00:34:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.888+0000 7f5b4ddd3740 20 client.15735 trim_cache size 0 max 16384 2024-04-25T00:34:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:34:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:34:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:34:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:34:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5b300a2cb0 msgr2=0x7f5b300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5b300a2cb0 0x7f5b300a5170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009ad0 tx=0x7f5b34003970 comp rx=0 tx=0).stop 2024-04-25T00:34:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 msgr2=0x7f5b4419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 0x7f5b4419ef00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d750 tx=0x7f5b3c00dc10 comp rx=0 tx=0).stop 2024-04-25T00:34:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 shutdown_connections 2024-04-25T00:34:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5b300a2cb0 0x7f5b300a5170 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b44102250 0x7f5b4419ef00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b44101890 0x7f5b44197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 --2- 172.21.15.31:0/4063271166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b441006a0 0x7f5b441979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 >> 172.21.15.31:0/4063271166 conn(0x7f5b440fbc10 msgr2=0x7f5b44104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 shutdown_connections 2024-04-25T00:34:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:34:52.889+0000 7f5b4b0bd640 1 -- 172.21.15.31:0/4063271166 wait complete. 2024-04-25T00:34:52.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:52 smithi040 ceph-mon[32945]: pgmap v490: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:34:52.898 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:52 smithi169 ceph-mon[33112]: pgmap v490: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:34:53.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:52 smithi031 ceph-mon[29167]: pgmap v490: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:34:54.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:54 smithi169 ceph-mon[33112]: pgmap v491: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:34:54.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:54 smithi040 ceph-mon[32945]: pgmap v491: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:34:55.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:54 smithi031 ceph-mon[29167]: pgmap v491: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:34:56.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:56 smithi169 ceph-mon[33112]: pgmap v492: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T00:34:56.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:56 smithi040 ceph-mon[32945]: pgmap v492: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T00:34:57.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:56 smithi031 ceph-mon[29167]: pgmap v492: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T00:34:57.732 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.732 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.732 INFO:tasks.workunit.client.0.smithi031.stdout:127/600: open fd = 3 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:128/600: open fd = 3 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:129/600: open fd = 3 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:130/600: open fd = 3 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.733 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:131/600: open fd = 3 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:132/600: open fd = 3 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:133/600: open fd = 3 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:134/600: open fd = 3 2024-04-25T00:34:57.734 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:135/600: open fd = 3 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:136/600: open fd = 3 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:137/600: open fd = 3 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:138/600: open fd = 3 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:139/600: open fd = 3 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:140/600: open fd = 3 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:141/600: open fd = 3 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.736 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:142/600: open fd = 3 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:143/600: open fd = 3 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:144/600: open fd = 3 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.737 INFO:tasks.workunit.client.0.smithi031.stdout:145/600: open fd = 3 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:146/600: open fd = 3 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:147/600: open fd = 3 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:148/600: open fd = 3 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.738 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:149/600: open fd = 3 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:150/600: open fd = 3 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:151/600: open fd = 3 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:152/600: open fd = 3 2024-04-25T00:34:57.739 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:153/600: open fd = 3 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:154/600: open fd = 3 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:155/600: open fd = 3 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.740 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:156/600: open fd = 3 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:157/600: open fd = 3 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:158/600: open fd = 3 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:159/600: open fd = 3 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.741 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:160/600: open fd = 3 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:161/600: open fd = 3 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:162/600: open fd = 3 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:163/600: open fd = 3 2024-04-25T00:34:57.742 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:164/600: open fd = 3 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:165/600: open fd = 3 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:166/600: open fd = 3 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.743 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:167/600: open fd = 3 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:168/600: open fd = 3 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:169/600: open fd = 3 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:170/600: open fd = 3 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:171/600: open fd = 3 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:172/600: open fd = 3 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:173/600: open fd = 3 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:174/600: open fd = 3 2024-04-25T00:34:57.745 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:175/600: open fd = 3 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:176/600: open fd = 3 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:177/600: open fd = 3 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.746 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:178/600: open fd = 3 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:179/600: open fd = 3 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:180/600: open fd = 3 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:181/600: open fd = 3 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.747 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:182/600: open fd = 3 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:183/600: open fd = 3 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:184/600: open fd = 3 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.748 INFO:tasks.workunit.client.0.smithi031.stdout:185/600: open fd = 3 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:186/600: open fd = 3 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:187/600: open fd = 3 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:188/600: open fd = 3 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:57.749 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:34:58.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:34:58 smithi169 ceph-mon[33112]: pgmap v493: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:34:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:34:58 smithi040 ceph-mon[32945]: pgmap v493: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:34:59.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:34:58 smithi031 ceph-mon[29167]: pgmap v493: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:35:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:00 smithi169 ceph-mon[33112]: pgmap v494: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:00 smithi040 ceph-mon[32945]: pgmap v494: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:00 smithi031 ceph-mon[29167]: pgmap v494: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:02.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:02 smithi169 ceph-mon[33112]: pgmap v495: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:02 smithi040 ceph-mon[32945]: pgmap v495: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:03.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:02 smithi031 ceph-mon[29167]: pgmap v495: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:04 smithi169 ceph-mon[33112]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:35:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:04 smithi040 ceph-mon[32945]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:35:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:04 smithi031 ceph-mon[29167]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.988+0000 7f44d9ca5640 1 -- 172.21.15.31:0/106538006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d41029c0 msgr2=0x7f44d4102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.988+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/106538006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d41029c0 0x7f44d4102e40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009900 tx=0x7f44c802a410 comp rx=0 tx=0).stop 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 -- 172.21.15.31:0/106538006 shutdown_connections 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/106538006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d4103380 0x7f44d4109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/106538006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d41029c0 0x7f44d4102e40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/106538006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d41017d0 0x7f44d4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 -- 172.21.15.31:0/106538006 >> 172.21.15.31:0/106538006 conn(0x7f44d40fcd60 msgr2=0x7f44d40ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 -- 172.21.15.31:0/106538006 shutdown_connections 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 -- 172.21.15.31:0/106538006 wait complete. 2024-04-25T00:35:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.989+0000 7f44d9ca5640 1 Processor -- start 2024-04-25T00:35:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 -- start start 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d41017d0 0x7f44d4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41029c0 0x7f44d4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 0x7f44d419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d360 con 0x7f44d4103380 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d1e0 con 0x7f44d41017d0 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d9ca5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d4e0 con 0x7f44d41029c0 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 0x7f44d419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 0x7f44d419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34956/0 (socket says 172.21.15.31:34956) 2024-04-25T00:35:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 -- 172.21.15.31:0/1227544115 learned_addr learned my addr 172.21.15.31:0/1227544115 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41029c0 0x7f44d4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d37fe640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d41017d0 0x7f44d4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 -- 172.21.15.31:0/1227544115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41029c0 msgr2=0x7f44d4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41029c0 0x7f44d4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 -- 172.21.15.31:0/1227544115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d41017d0 msgr2=0x7f44d4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d41017d0 0x7f44d4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 -- 172.21.15.31:0/1227544115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419f600 con 0x7f44d4103380 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.990+0000 7f44d3fff640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 0x7f44d419ef90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b970 tx=0x7f44cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.991+0000 7f44d0ff9640 1 -- 172.21.15.31:0/1227544115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc019070 con 0x7f44d4103380 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.991+0000 7f44d0ff9640 1 -- 172.21.15.31:0/1227544115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44cc014440 con 0x7f44d4103380 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.991+0000 7f44d0ff9640 1 -- 172.21.15.31:0/1227544115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0075d0 con 0x7f44d4103380 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.991+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d419f8f0 con 0x7f44d4103380 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.991+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d419fdc0 con 0x7f44d4103380 2024-04-25T00:35:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.993+0000 7f44d0ff9640 1 -- 172.21.15.31:0/1227544115 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f44cc01c070 con 0x7f44d4103380 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.993+0000 7f44dc9b5740 1 Processor -- start 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.993+0000 7f44dc9b5740 1 -- start start 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf549680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 0x5638cf54e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf551250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5638cf4a8cb0 con 0x5638cf549c50 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5638cf4515b0 con 0x5638cf54edc0 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5638cf4a4c70 con 0x5638cf549260 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 0x5638cf54e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 0x5638cf54e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34960/0 (socket says 172.21.15.31:34960) 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 -- 172.21.15.31:0/3419653318 learned_addr learned my addr 172.21.15.31:0/3419653318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d0ff9640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f44bc0a2c30 0x7f44bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf549680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf551250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d3fff640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f44bc0a2c30 0x7f44bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 -- 172.21.15.31:0/3419653318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 msgr2=0x5638cf549680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf549680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 -- 172.21.15.31:0/3419653318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 msgr2=0x5638cf551250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf551250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d37fe640 1 -- 172.21.15.31:0/3419653318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cf1bd210 con 0x5638cf549c50 2024-04-25T00:35:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf551250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.994+0000 7f44d0ff9640 1 -- 172.21.15.31:0/1227544115 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f44cc0ce650 con 0x7f44d4103380 2024-04-25T00:35:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 0x5638cf54e750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44c400b7d0 tx=0x7f44c4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08f2651a9ff79f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44b9ffb640 1 -- 172.21.15.31:0/3419653318 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4003dd0 con 0x5638cf549c50 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44d3fff640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f44bc0a2c30 0x7f44bc0a50f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f44cc097330 tx=0x7f44cc004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44b9ffb640 1 -- 172.21.15.31:0/3419653318 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c4015070 con 0x5638cf549c50 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44b9ffb640 1 -- 172.21.15.31:0/3419653318 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c40116a0 con 0x5638cf549c50 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3419653318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 msgr2=0x5638cf54e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 0x5638cf54e750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44c400b7d0 tx=0x7f44c4004610 comp rx=0 tx=0).stop 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3419653318 shutdown_connections 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf551250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549c50 0x5638cf54e750 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3419653318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf549680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.995+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3419653318 >> 172.21.15.31:0/3419653318 conn(0x5638cf5404c0 msgr2=0x5638cf5408c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3419653318 shutdown_connections 2024-04-25T00:35:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3419653318 wait complete. 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 -- start start 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 Processor -- start 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf553bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf55ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5638cf551d70 con 0x5638cf549260 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5638cf464b50 con 0x5638cf54edc0 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.996+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5638cf551ef0 con 0x5638cf549c50 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44dc9b5740 10 client.0 _ll_register_callbacks cb 0x5638cf49fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34970/0 (socket says 172.21.15.31:34970) 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1784657432 learned_addr learned my addr 172.21.15.31:0/1784657432 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d37fe640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf553bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf55ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 msgr2=0x5638cf553bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf553bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1784657432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 msgr2=0x5638cf55ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf55ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1784657432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cf289620 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553680 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f44c802a930 tx=0x7f44c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324910f3eccc7969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44b2ffd640 1 -- 172.21.15.31:0/1784657432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8035040 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44b2ffd640 1 -- 172.21.15.31:0/1784657432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f44c80035f0 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.997+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c4015070 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44b2ffd640 1 -- 172.21.15.31:0/1784657432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44c8051050 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 10 client.15747 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638cf55df50 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44b2ffd640 1 -- 172.21.15.31:0/1784657432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c4015070 con 0x5638cf549260 2024-04-25T00:35:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44b2ffd640 1 -- 172.21.15.31:0/1784657432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44c804a060 con 0x5638cf549260 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 10 client.15747 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 20 client.15747 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf55f620 0x5638cf561a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 4 client.15747 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638cf4b61f0 con 0x5638cf55f620 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.998+0000 7f44d37fe640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf55f620 0x5638cf561a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.999+0000 7f44b2ffd640 10 client.15747 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:05.999+0000 7f44d37fe640 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf55f620 0x5638cf561a50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6dc04b41204d480 server_cookie=f1afc9b5d10c3aa8 in_seq=0 out_seq=0 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44b2ffd640 1 -- 172.21.15.31:0/1784657432 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638cf4b61f0 con 0x5638cf55f620 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44b2ffd640 10 client.15747 handle_command_reply: tid=0 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 client.15747 shutdown 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 msgr2=0x5638cf553680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553680 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f44c802a930 tx=0x7f44c8009260 comp rx=0 tx=0).stop 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 shutdown_connections 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf55f620 0x5638cf561a50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf55ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf553bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1784657432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553680 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44b2ffd640 0 client.15747 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 >> 172.21.15.31:0/1784657432 conn(0x5638cf5404c0 msgr2=0x5638cf53f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 shutdown_connections 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1784657432 wait complete. 2024-04-25T00:35:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.000+0000 7f44dc9b5740 20 client.15747 trim_cache size 0 max 16384 2024-04-25T00:35:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 Processor -- start 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 -- start start 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf55ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf54e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a6a80 con 0x5638cf549260 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a85c0 con 0x5638cf54edc0 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a7150 con 0x5638cf549c50 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34982/0 (socket says 172.21.15.31:34982) 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.015+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1341926779 learned_addr learned my addr 172.21.15.31:0/1341926779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d37fe640 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf55ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1341926779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 msgr2=0x5638cf55ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf55ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1341926779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 msgr2=0x5638cf54e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf54e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d2ffd640 1 -- 172.21.15.31:0/1341926779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cf540000 con 0x5638cf549260 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553b20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f44c803d760 tx=0x7f44c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe468e759b4f6712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44b97fa640 1 -- 172.21.15.31:0/1341926779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8035040 con 0x5638cf549260 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44b97fa640 1 -- 172.21.15.31:0/1341926779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c8005850 con 0x5638cf549260 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.016+0000 7f44b97fa640 1 -- 172.21.15.31:0/1341926779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80035f0 con 0x5638cf549260 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1341926779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 msgr2=0x5638cf553b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553b20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f44c803d760 tx=0x7f44c8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1341926779 shutdown_connections 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf54edc0 0x5638cf54e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549c50 0x5638cf55ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/1341926779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf549260 0x5638cf553b20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1341926779 >> 172.21.15.31:0/1341926779 conn(0x5638cf5404c0 msgr2=0x5638cf543470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1341926779 shutdown_connections 2024-04-25T00:35:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 -- 172.21.15.31:0/1341926779 wait complete. 2024-04-25T00:35:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 -- start start 2024-04-25T00:35:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.017+0000 7f44dc9b5740 1 Processor -- start 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf5584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf549c50 0x5638cf5589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 0x5638cf559a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a7ef0 con 0x5638cf54edc0 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a7820 con 0x5638cf549c50 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a4a60 con 0x5638cf549260 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 10 client.0 _ll_register_callbacks cb 0x7f44c8032b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 0x5638cf559a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 0x5638cf559a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34994/0 (socket says 172.21.15.31:34994) 2024-04-25T00:35:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 -- 172.21.15.31:0/3548469620 learned_addr learned my addr 172.21.15.31:0/3548469620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf5584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf549c50 0x5638cf5589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 -- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 msgr2=0x5638cf5584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf5584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 -- 172.21.15.31:0/3548469620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf549c50 msgr2=0x5638cf5589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf549c50 0x5638cf5589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 -- 172.21.15.31:0/3548469620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cf559ff0 con 0x5638cf54edc0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf549c50 0x5638cf5589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44d2ffd640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 0x5638cf559a00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009690 tx=0x7f44c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a994f65f3f248f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44b2ffd640 1 -- 172.21.15.31:0/3548469620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80037f0 con 0x5638cf54edc0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44b2ffd640 1 -- 172.21.15.31:0/3548469620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f44c8003990 con 0x5638cf54edc0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.018+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c8005850 con 0x5638cf54edc0 2024-04-25T00:35:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44b2ffd640 1 -- 172.21.15.31:0/3548469620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44c8003b90 con 0x5638cf54edc0 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638cf5c5e10 con 0x5638cf54edc0 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44b2ffd640 1 -- 172.21.15.31:0/3548469620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c8005850 con 0x5638cf54edc0 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44b2ffd640 1 -- 172.21.15.31:0/3548469620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44c804a070 con 0x5638cf54edc0 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 20 client.15753 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf5c6690 0x5638cf5c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 4 client.15753 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638cf4b0540 con 0x5638cf5c6690 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.019+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf5c6690 0x5638cf5c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.020+0000 7f44d37fe640 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf5c6690 0x5638cf5c8ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2dab6caff02744 server_cookie=753320ea8aa143bf in_seq=0 out_seq=0 2024-04-25T00:35:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.020+0000 7f44b2ffd640 10 client.15753 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44b2ffd640 1 -- 172.21.15.31:0/3548469620 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638cf4b0540 con 0x5638cf5c6690 2024-04-25T00:35:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44b2ffd640 10 client.15753 handle_command_reply: tid=0 2024-04-25T00:35:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 client.15753 shutdown 2024-04-25T00:35:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 msgr2=0x5638cf559a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 0x5638cf559a00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009690 tx=0x7f44c802a410 comp rx=0 tx=0).stop 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 shutdown_connections 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5638cf5c6690 0x5638cf5c8ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5638cf54edc0 0x5638cf559a00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638cf549c50 0x5638cf5589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44b2ffd640 0 client.15753 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 --2- 172.21.15.31:0/3548469620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5638cf549260 0x5638cf5584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 >> 172.21.15.31:0/3548469620 conn(0x5638cf5404c0 msgr2=0x5638cf543470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 shutdown_connections 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.035+0000 7f44dc9b5740 1 -- 172.21.15.31:0/3548469620 wait complete. 2024-04-25T00:35:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44dc9b5740 20 client.15753 trim_cache size 0 max 16384 2024-04-25T00:35:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:35:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:35:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:35:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f44bc0a2c30 msgr2=0x7f44bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f44bc0a2c30 0x7f44bc0a50f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f44cc097330 tx=0x7f44cc004b50 comp rx=0 tx=0).stop 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 msgr2=0x7f44d419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 0x7f44d419ef90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b970 tx=0x7f44cc00be30 comp rx=0 tx=0).stop 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 shutdown_connections 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f44bc0a2c30 0x7f44bc0a50f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d4103380 0x7f44d419ef90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41029c0 0x7f44d4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 --2- 172.21.15.31:0/1227544115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d41017d0 0x7f44d4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 >> 172.21.15.31:0/1227544115 conn(0x7f44d40fcd60 msgr2=0x7f44d4105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 shutdown_connections 2024-04-25T00:35:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:06.036+0000 7f44d9ca5640 1 -- 172.21.15.31:0/1227544115 wait complete. 2024-04-25T00:35:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:06 smithi169 ceph-mon[33112]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:06 smithi040 ceph-mon[32945]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:07.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:06 smithi031 ceph-mon[29167]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:08.824 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:08 smithi031 ceph-mon[29167]: pgmap v498: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:08.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:08 smithi169 ceph-mon[33112]: pgmap v498: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:08 smithi040 ceph-mon[32945]: pgmap v498: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:10 smithi169 ceph-mon[33112]: pgmap v499: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-25T00:35:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:10 smithi040 ceph-mon[32945]: pgmap v499: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-25T00:35:11.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:10 smithi031 ceph-mon[29167]: pgmap v499: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-25T00:35:12.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:12 smithi169 ceph-mon[33112]: pgmap v500: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:12 smithi040 ceph-mon[32945]: pgmap v500: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:13.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:12 smithi031 ceph-mon[29167]: pgmap v500: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:14.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:14 smithi169 ceph-mon[33112]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:14.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:14 smithi040 ceph-mon[32945]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:14 smithi031 ceph-mon[29167]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:16.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:16 smithi169 ceph-mon[33112]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:16 smithi040 ceph-mon[32945]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:16 smithi031 ceph-mon[29167]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:18.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:18 smithi169 ceph-mon[33112]: pgmap v503: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:18 smithi040 ceph-mon[32945]: pgmap v503: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:18 smithi031 ceph-mon[29167]: pgmap v503: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 -- 172.21.15.31:0/1121301765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 msgr2=0x7f53f8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 --2- 172.21.15.31:0/1121301765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f8101bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009900 tx=0x7f53f402a410 comp rx=0 tx=0).stop 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 -- 172.21.15.31:0/1121301765 shutdown_connections 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 --2- 172.21.15.31:0/1121301765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8103350 0x7f53f8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 --2- 172.21.15.31:0/1121301765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53f8102990 0x7f53f8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 --2- 172.21.15.31:0/1121301765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f8101bc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.135+0000 7f53fffee640 1 -- 172.21.15.31:0/1121301765 >> 172.21.15.31:0/1121301765 conn(0x7f53f80fcd30 msgr2=0x7f53f80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 -- 172.21.15.31:0/1121301765 shutdown_connections 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 -- 172.21.15.31:0/1121301765 wait complete. 2024-04-25T00:35:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 Processor -- start 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 -- start start 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53f8102990 0x7f53f8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8103350 0x7f53f8074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f53f810d2c0 con 0x7f53f81017a0 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53f810d140 con 0x7f53f8103350 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.136+0000 7f53fffee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53f810d440 con 0x7f53f8102990 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f80754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54556/0 (socket says 172.21.15.31:54556) 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3831110331 learned_addr learned my addr 172.21.15.31:0/3831110331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fd589640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53f8102990 0x7f53f8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fe58b640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8103350 0x7f53f8074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3831110331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53f8102990 msgr2=0x7f53f8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53f8102990 0x7f53f8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3831110331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8103350 msgr2=0x7f53f8074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8103350 0x7f53f8074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3831110331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8074900 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f80754f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f53f40314e0 tx=0x7f53f40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53eeffd640 1 -- 172.21.15.31:0/3831110331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f403f030 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53eeffd640 1 -- 172.21.15.31:0/3831110331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f4003cb0 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81a1a20 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53eeffd640 1 -- 172.21.15.31:0/3831110331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4035070 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.137+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81a1f70 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.140+0000 7f53eeffd640 1 -- 172.21.15.31:0/3831110331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f53f4042030 con 0x7f53f81017a0 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.140+0000 7f5402dde740 1 Processor -- start 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.140+0000 7f5402dde740 1 -- start start 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f5402dde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c819270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f5402dde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c81e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f5402dde740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c820e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f5402dde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556d9c77ccb0 con 0x556d9c818e50 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f5402dde740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556d9c7255b0 con 0x556d9c81e9b0 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f5402dde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d9c778c70 con 0x556d9c819840 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c819270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c819270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54566/0 (socket says 172.21.15.31:54566) 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 -- 172.21.15.31:0/835031220 learned_addr learned my addr 172.21.15.31:0/835031220 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c81e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53eeffd640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53e40a2c30 0x7f53e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fd589640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c820e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53e40a2c30 0x7f53e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 -- 172.21.15.31:0/835031220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 msgr2=0x556d9c81e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c81e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 -- 172.21.15.31:0/835031220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 msgr2=0x556d9c820e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c820e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 -- 172.21.15.31:0/835031220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9c491210 con 0x556d9c818e50 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fd589640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c820e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53eeffd640 1 -- 172.21.15.31:0/3831110331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f53f40ef6b0 con 0x7f53f81017a0 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c81e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.141+0000 7f53fe58b640 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c819270 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f53f0013a70 tx=0x7f53f0013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82692c778454e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f53e3fff640 1 -- 172.21.15.31:0/835031220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0013540 con 0x556d9c818e50 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f53e3fff640 1 -- 172.21.15.31:0/835031220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f0022070 con 0x556d9c818e50 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f53e3fff640 1 -- 172.21.15.31:0/835031220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f001cb80 con 0x556d9c818e50 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f53fe58b640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53e40a2c30 0x7f53e40a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53f0021000 tx=0x7f53f001f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 -- 172.21.15.31:0/835031220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 msgr2=0x556d9c819270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c819270 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f53f0013a70 tx=0x7f53f0013f30 comp rx=0 tx=0).stop 2024-04-25T00:35:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 -- 172.21.15.31:0/835031220 shutdown_connections 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c820e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c81e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 --2- 172.21.15.31:0/835031220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c819270 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.142+0000 7f5402dde740 1 -- 172.21.15.31:0/835031220 >> 172.21.15.31:0/835031220 conn(0x556d9c8144c0 msgr2=0x556d9c8148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 -- 172.21.15.31:0/835031220 shutdown_connections 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 -- 172.21.15.31:0/835031220 wait complete. 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 -- start start 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 Processor -- start 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 0x556d9c823820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c81e9b0 0x556d9c82a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556d9c821960 con 0x556d9c819840 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556d9c738b50 con 0x556d9c818e50 2024-04-25T00:35:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d9c821ae0 con 0x556d9c81e9b0 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.143+0000 7f5402dde740 10 client.0 _ll_register_callbacks cb 0x556d9c773ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 0x556d9c823820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 0x556d9c823820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54568/0 (socket says 172.21.15.31:54568) 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 -- 172.21.15.31:0/2642615584 learned_addr learned my addr 172.21.15.31:0/2642615584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fe58b640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c81e9b0 0x556d9c82a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 -- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c81e9b0 msgr2=0x556d9c82a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c81e9b0 0x556d9c82a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 -- 172.21.15.31:0/2642615584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 msgr2=0x556d9c8232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 -- 172.21.15.31:0/2642615584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9c55d620 con 0x556d9c819840 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fe58b640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c81e9b0 0x556d9c82a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53fd589640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 0x556d9c823820 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b9c0 tx=0x7f53e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0974b0af76366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53e17fa640 1 -- 172.21.15.31:0/2642615584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e801b440 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53e17fa640 1 -- 172.21.15.31:0/2642615584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f53e801b5e0 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f0022070 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f53e17fa640 1 -- 172.21.15.31:0/2642615584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53e8005020 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f5402dde740 10 client.15765 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.144+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556d9c82dca0 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f53e17fa640 1 -- 172.21.15.31:0/2642615584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f0022070 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f53e17fa640 1 -- 172.21.15.31:0/2642615584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53e8019020 con 0x556d9c819840 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f5402dde740 10 client.15765 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f5402dde740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f5402dde740 20 client.15765 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f5402dde740 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c82f370 0x556d9c8317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f5402dde740 4 client.15765 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.145+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9c78a1f0 con 0x556d9c82f370 2024-04-25T00:35:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.146+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c82f370 0x556d9c8317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.147+0000 7f53e17fa640 10 client.15765 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.147+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c82f370 0x556d9c8317e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9653ddae5f4aa3 server_cookie=746faaf04cc23ab3 in_seq=0 out_seq=0 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f53e17fa640 1 -- 172.21.15.31:0/2642615584 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d9c78a1f0 con 0x556d9c82f370 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f53e17fa640 10 client.15765 handle_command_reply: tid=0 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 client.15765 shutdown 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 msgr2=0x556d9c823820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 0x556d9c823820 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b9c0 tx=0x7f53e800be80 comp rx=0 tx=0).stop 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 shutdown_connections 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c82f370 0x556d9c8317e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c81e9b0 0x556d9c82a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c819840 0x556d9c823820 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 --2- 172.21.15.31:0/2642615584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f53e17fa640 0 client.15765 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 >> 172.21.15.31:0/2642615584 conn(0x556d9c8144c0 msgr2=0x556d9c8135c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 shutdown_connections 2024-04-25T00:35:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 1 -- 172.21.15.31:0/2642615584 wait complete. 2024-04-25T00:35:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.148+0000 7f5402dde740 20 client.15765 trim_cache size 0 max 16384 2024-04-25T00:35:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 Processor -- start 2024-04-25T00:35:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 -- start start 2024-04-25T00:35:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c812bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c82a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c8130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81aad60 con 0x556d9c818e50 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81ac8a0 con 0x556d9c81e9b0 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f5402dde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81ab430 con 0x556d9c819840 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.163+0000 7f53fdd8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c812bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c812bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54578/0 (socket says 172.21.15.31:54578) 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3262285204 learned_addr learned my addr 172.21.15.31:0/3262285204 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fd589640 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c82a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fe58b640 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c8130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3262285204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 msgr2=0x556d9c82a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c82a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3262285204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 msgr2=0x556d9c8130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c8130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 -- 172.21.15.31:0/3262285204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9c814060 con 0x556d9c818e50 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fd589640 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c82a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c812bb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f53f40b8320 tx=0x7f53f40dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887f6d8aa9e5e117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53e37fe640 1 -- 172.21.15.31:0/3262285204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4049040 con 0x556d9c818e50 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53e37fe640 1 -- 172.21.15.31:0/3262285204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f40ba1c0 con 0x556d9c818e50 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.164+0000 7f53e37fe640 1 -- 172.21.15.31:0/3262285204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f403b460 con 0x556d9c818e50 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 -- 172.21.15.31:0/3262285204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 msgr2=0x556d9c812bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c812bb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f53f40b8320 tx=0x7f53f40dc040 comp rx=0 tx=0).stop 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 -- 172.21.15.31:0/3262285204 shutdown_connections 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c81e9b0 0x556d9c8130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c82a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 --2- 172.21.15.31:0/3262285204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c818e50 0x556d9c812bb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 -- 172.21.15.31:0/3262285204 >> 172.21.15.31:0/3262285204 conn(0x556d9c8144c0 msgr2=0x556d9c8148a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 -- 172.21.15.31:0/3262285204 shutdown_connections 2024-04-25T00:35:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 -- 172.21.15.31:0/3262285204 wait complete. 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 -- start start 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.165+0000 7f5402dde740 1 Processor -- start 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c828930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 0x556d9c829880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81ac1d0 con 0x556d9c81e9b0 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81abb00 con 0x556d9c818e50 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81a8d40 con 0x556d9c819840 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 10 client.0 _ll_register_callbacks cb 0x7f53e8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 0x556d9c829880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 0x556d9c829880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54592/0 (socket says 172.21.15.31:54592) 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 -- 172.21.15.31:0/845450858 learned_addr learned my addr 172.21.15.31:0/845450858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fd589640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c828930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 -- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 msgr2=0x556d9c828930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c828930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 -- 172.21.15.31:0/845450858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 msgr2=0x556d9c8283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8283f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 -- 172.21.15.31:0/845450858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9c829e70 con 0x556d9c81e9b0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53fe58b640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 0x556d9c829880 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f53f001cf80 tx=0x7f53f0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b46a49a4afa97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53e17fa640 1 -- 172.21.15.31:0/845450858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0013420 con 0x556d9c81e9b0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f53e17fa640 1 -- 172.21.15.31:0/845450858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f53f00135c0 con 0x556d9c81e9b0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.166+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f40ba1c0 con 0x556d9c81e9b0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f53e17fa640 1 -- 172.21.15.31:0/845450858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53f0003970 con 0x556d9c81e9b0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 10 client.15771 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556d9c895cf0 con 0x556d9c81e9b0 2024-04-25T00:35:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f53e17fa640 1 -- 172.21.15.31:0/845450858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f40ba1c0 con 0x556d9c81e9b0 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f53e17fa640 1 -- 172.21.15.31:0/845450858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53f002f420 con 0x556d9c81e9b0 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 20 client.15771 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c896490 0x556d9c8988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 4 client.15771 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d9c784540 con 0x556d9c896490 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.167+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c896490 0x556d9c8988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.168+0000 7f53fdd8a640 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c896490 0x556d9c8988b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0fb229b8b76bd1 server_cookie=65330110da4b7051 in_seq=0 out_seq=0 2024-04-25T00:35:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.168+0000 7f53e17fa640 10 client.15771 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.182+0000 7f53e17fa640 1 -- 172.21.15.31:0/845450858 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d9c784540 con 0x556d9c896490 2024-04-25T00:35:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.182+0000 7f53e17fa640 10 client.15771 handle_command_reply: tid=0 2024-04-25T00:35:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.182+0000 7f5402dde740 1 client.15771 shutdown 2024-04-25T00:35:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 msgr2=0x556d9c829880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 0x556d9c829880 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f53f001cf80 tx=0x7f53f0009880 comp rx=0 tx=0).stop 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 shutdown_connections 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556d9c896490 0x556d9c8988b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d9c81e9b0 0x556d9c829880 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d9c819840 0x556d9c828930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 --2- 172.21.15.31:0/845450858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d9c818e50 0x556d9c8283f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f53e17fa640 0 client.15771 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 >> 172.21.15.31:0/845450858 conn(0x556d9c8144c0 msgr2=0x556d9c817240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 shutdown_connections 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 1 -- 172.21.15.31:0/845450858 wait complete. 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.183+0000 7f5402dde740 20 client.15771 trim_cache size 0 max 16384 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:35:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:35:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:35:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:35:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53e40a2c30 msgr2=0x7f53e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53e40a2c30 0x7f53e40a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53f0021000 tx=0x7f53f001f000 comp rx=0 tx=0).stop 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 msgr2=0x7f53f80754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f80754f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f53f40314e0 tx=0x7f53f40042e0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 shutdown_connections 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8103350 0x7f53f8074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53f8102990 0x7f53f8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f53e40a2c30 0x7f53e40a50f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 --2- 172.21.15.31:0/3831110331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f81017a0 0x7f53f80754f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 >> 172.21.15.31:0/3831110331 conn(0x7f53f80fcd30 msgr2=0x7f53f8105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 shutdown_connections 2024-04-25T00:35:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:19.184+0000 7f53fffee640 1 -- 172.21.15.31:0/3831110331 wait complete. 2024-04-25T00:35:20.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:20 smithi169 ceph-mon[33112]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:20 smithi040 ceph-mon[32945]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:20 smithi031 ceph-mon[29167]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:22 smithi169 ceph-mon[33112]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:35:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:22 smithi040 ceph-mon[32945]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:35:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:22 smithi031 ceph-mon[29167]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:35:24.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:24 smithi169 ceph-mon[33112]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:24 smithi040 ceph-mon[32945]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:25.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:24 smithi031 ceph-mon[29167]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:26.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:26 smithi169 ceph-mon[33112]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:26 smithi040 ceph-mon[32945]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:27.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:26 smithi031 ceph-mon[29167]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:28 smithi169 ceph-mon[33112]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:35:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:28 smithi040 ceph-mon[32945]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:35:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:28 smithi031 ceph-mon[29167]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:35:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:30 smithi169 ceph-mon[33112]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:30 smithi040 ceph-mon[32945]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:31.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:30 smithi031 ceph-mon[29167]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 -- 172.21.15.31:0/1076037641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc1039d0 msgr2=0x7f61bc103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 --2- 172.21.15.31:0/1076037641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc1039d0 0x7f61bc103df0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009900 tx=0x7f61b802a410 comp rx=0 tx=0).stop 2024-04-25T00:35:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 -- 172.21.15.31:0/1076037641 shutdown_connections 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 --2- 172.21.15.31:0/1076037641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc105580 0x7f61bc10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 --2- 172.21.15.31:0/1076037641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61bc104bc0 0x7f61bc105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 --2- 172.21.15.31:0/1076037641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc1039d0 0x7f61bc103df0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 -- 172.21.15.31:0/1076037641 >> 172.21.15.31:0/1076037641 conn(0x7f61bc0fef60 msgr2=0x7f61bc101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 -- 172.21.15.31:0/1076037641 shutdown_connections 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.283+0000 7f61c3514640 1 -- 172.21.15.31:0/1076037641 wait complete. 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 Processor -- start 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 -- start start 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61bc1039d0 0x7f61bc199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc104bc0 0x7f61bc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 0x7f61bc1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc10f550 con 0x7f61bc105580 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc10f3d0 con 0x7f61bc1039d0 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c3514640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc10f6d0 con 0x7f61bc104bc0 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c1ab1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 0x7f61bc1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c1ab1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 0x7f61bc1a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34836/0 (socket says 172.21.15.31:34836) 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c1ab1640 1 -- 172.21.15.31:0/2189155871 learned_addr learned my addr 172.21.15.31:0/2189155871 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c12b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61bc1039d0 0x7f61bc199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.284+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc104bc0 0x7f61bc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c1ab1640 1 -- 172.21.15.31:0/2189155871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc104bc0 msgr2=0x7f61bc19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc104bc0 0x7f61bc19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c1ab1640 1 -- 172.21.15.31:0/2189155871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61bc1039d0 msgr2=0x7f61bc199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61bc1039d0 0x7f61bc199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c1ab1640 1 -- 172.21.15.31:0/2189155871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc1a1750 con 0x7f61bc105580 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 0x7f61bc1a10e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f61b400eaa0 tx=0x7f61b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61b27fc640 1 -- 172.21.15.31:0/2189155871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b400cdb0 con 0x7f61bc105580 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61b27fc640 1 -- 172.21.15.31:0/2189155871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b4005020 con 0x7f61bc105580 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc1a1a40 con 0x7f61bc105580 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61b27fc640 1 -- 172.21.15.31:0/2189155871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40116d0 con 0x7f61bc105580 2024-04-25T00:35:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.285+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a1fe0 con 0x7f61bc105580 2024-04-25T00:35:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.287+0000 7f61b27fc640 1 -- 172.21.15.31:0/2189155871 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f61b40051d0 con 0x7f61bc105580 2024-04-25T00:35:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 Processor -- start 2024-04-25T00:35:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 -- start start 2024-04-25T00:35:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61b27fc640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61a80a2cb0 0x7f61a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd46270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 0x55bb7dd4b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd4de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb7dca9cb0 con 0x55bb7dd46840 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb7dc525b0 con 0x55bb7dd45e50 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c6224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bb7dca5c70 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61a80a2cb0 0x7f61a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c0aaf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 0x55bb7dd4b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c0aaf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 0x55bb7dd4b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34838/0 (socket says 172.21.15.31:34838) 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61c0aaf640 1 -- 172.21.15.31:0/1162522903 learned_addr learned my addr 172.21.15.31:0/1162522903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.288+0000 7f61b27fc640 1 -- 172.21.15.31:0/2189155871 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f61b40cf110 con 0x7f61bc105580 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c12b0640 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd46270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd4de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 -- 172.21.15.31:0/1162522903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 msgr2=0x55bb7dd4de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd4de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 -- 172.21.15.31:0/1162522903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 msgr2=0x55bb7dd46270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd46270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 -- 172.21.15.31:0/1162522903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7d9be210 con 0x55bb7dd46840 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 0x55bb7dd4b340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61ac011a10 tx=0x7f61ac011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59b9102ba2f8d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61a80a2cb0 0x7f61a80a5170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e780 tx=0x7f61ac016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61a77fe640 1 -- 172.21.15.31:0/1162522903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00ae90 con 0x55bb7dd46840 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61a77fe640 1 -- 172.21.15.31:0/1162522903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61ac023070 con 0x55bb7dd46840 2024-04-25T00:35:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.289+0000 7f61a77fe640 1 -- 172.21.15.31:0/1162522903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac01e680 con 0x55bb7dd46840 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 -- 172.21.15.31:0/1162522903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 msgr2=0x55bb7dd4b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 0x55bb7dd4b340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61ac011a10 tx=0x7f61ac011e10 comp rx=0 tx=0).stop 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 -- 172.21.15.31:0/1162522903 shutdown_connections 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd4de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd46840 0x55bb7dd4b340 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 --2- 172.21.15.31:0/1162522903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd46270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 -- 172.21.15.31:0/1162522903 >> 172.21.15.31:0/1162522903 conn(0x55bb7dd414c0 msgr2=0x55bb7dd418c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 -- 172.21.15.31:0/1162522903 shutdown_connections 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 -- 172.21.15.31:0/1162522903 wait complete. 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 -- start start 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.290+0000 7f61c6224740 1 Processor -- start 2024-04-25T00:35:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 0x55bb7dd58ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd58ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd60030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb7dd4e900 con 0x55bb7dd45e50 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb7dc65b50 con 0x55bb7dd46840 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bb7dd4ea80 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c6224740 10 client.0 _ll_register_callbacks cb 0x55bb7dca0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 0x55bb7dd58ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 0x55bb7dd58ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34842/0 (socket says 172.21.15.31:34842) 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 -- 172.21.15.31:0/743760624 learned_addr learned my addr 172.21.15.31:0/743760624 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd58ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd60030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 -- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 msgr2=0x55bb7dd60030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd60030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 -- 172.21.15.31:0/743760624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 msgr2=0x55bb7dd58ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd58ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.291+0000 7f61c12b0640 1 -- 172.21.15.31:0/743760624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7da8a620 con 0x55bb7dd45e50 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c12b0640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 0x55bb7dd58ab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009900 tx=0x7f61b8004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce612982e0576d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61a4ff9640 1 -- 172.21.15.31:0/743760624 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8035040 con 0x55bb7dd45e50 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61ac023070 con 0x55bb7dd45e50 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61a4ff9640 1 -- 172.21.15.31:0/743760624 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f61b8003bc0 con 0x55bb7dd45e50 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61a4ff9640 1 -- 172.21.15.31:0/743760624 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61b8003dc0 con 0x55bb7dd45e50 2024-04-25T00:35:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 10 client.15783 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb7dd63410 con 0x55bb7dd45e50 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61a4ff9640 1 -- 172.21.15.31:0/743760624 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61ac023070 con 0x55bb7dd45e50 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61a4ff9640 1 -- 172.21.15.31:0/743760624 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61b804f070 con 0x55bb7dd45e50 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 10 client.15783 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 20 client.15783 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7dd64b00 0x55bb7dd66f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 4 client.15783 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.292+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7dcb71f0 con 0x55bb7dd64b00 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.293+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7dd64b00 0x55bb7dd66f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.293+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7dd64b00 0x55bb7dd66f70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4ec61f3c2879e8 server_cookie=e9c33afaf9a13f79 in_seq=0 out_seq=0 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.293+0000 7f61a4ff9640 10 client.15783 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61a4ff9640 1 -- 172.21.15.31:0/743760624 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb7dcb71f0 con 0x55bb7dd64b00 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61a4ff9640 10 client.15783 handle_command_reply: tid=0 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 client.15783 shutdown 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 msgr2=0x55bb7dd58ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 0x55bb7dd58ab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009900 tx=0x7f61b8004580 comp rx=0 tx=0).stop 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 shutdown_connections 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7dd64b00 0x55bb7dd66f70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd58ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 --2- 172.21.15.31:0/743760624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd45e50 0x55bb7dd58ab0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61a4ff9640 0 client.15783 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 >> 172.21.15.31:0/743760624 conn(0x55bb7dd414c0 msgr2=0x55bb7dd405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 shutdown_connections 2024-04-25T00:35:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 1 -- 172.21.15.31:0/743760624 wait complete. 2024-04-25T00:35:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.294+0000 7f61c6224740 20 client.15783 trim_cache size 0 max 16384 2024-04-25T00:35:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 Processor -- start 2024-04-25T00:35:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 -- start start 2024-04-25T00:35:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd59010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd46840 0x55bb7dd5ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a68c0 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a8400 con 0x55bb7dd45e50 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.309+0000 7f61c6224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a6f90 con 0x55bb7dd46840 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd44950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34856/0 (socket says 172.21.15.31:34856) 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 -- 172.21.15.31:0/1383541484 learned_addr learned my addr 172.21.15.31:0/1383541484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c12b0640 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd59010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd46840 0x55bb7dd5ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 -- 172.21.15.31:0/1383541484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd46840 msgr2=0x55bb7dd5ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd46840 0x55bb7dd5ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 -- 172.21.15.31:0/1383541484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 msgr2=0x55bb7dd59010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd59010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 -- 172.21.15.31:0/1383541484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7dd40d40 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c12b0640 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd59010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd44950 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61b40983c0 tx=0x7f61b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11124dbbcd11ff82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61a6ffd640 1 -- 172.21.15.31:0/1383541484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b401f040 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61a6ffd640 1 -- 172.21.15.31:0/1383541484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b40a45b0 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.310+0000 7f61a6ffd640 1 -- 172.21.15.31:0/1383541484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4098060 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 -- 172.21.15.31:0/1383541484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 msgr2=0x55bb7dd44950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd44950 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61b40983c0 tx=0x7f61b4005830 comp rx=0 tx=0).stop 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 -- 172.21.15.31:0/1383541484 shutdown_connections 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd44950 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd46840 0x55bb7dd5ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 --2- 172.21.15.31:0/1383541484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd45e50 0x55bb7dd59010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 -- 172.21.15.31:0/1383541484 >> 172.21.15.31:0/1383541484 conn(0x55bb7dd414c0 msgr2=0x55bb7dd44240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 -- 172.21.15.31:0/1383541484 shutdown_connections 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 -- 172.21.15.31:0/1383541484 wait complete. 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 -- start start 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.311+0000 7f61c6224740 1 Processor -- start 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd45e50 0x55bb7dd5daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd5dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a7d30 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a7660 con 0x55bb7dd46840 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a48a0 con 0x55bb7dd45e50 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c6224740 10 client.0 _ll_register_callbacks cb 0x7f61b8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd5eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34868/0 (socket says 172.21.15.31:34868) 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 -- 172.21.15.31:0/3266748781 learned_addr learned my addr 172.21.15.31:0/3266748781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd5dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c12b0640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd45e50 0x55bb7dd5daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 -- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd45e50 msgr2=0x55bb7dd5daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd45e50 0x55bb7dd5daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 -- 172.21.15.31:0/3266748781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 msgr2=0x55bb7dd5dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd5dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 -- 172.21.15.31:0/3266748781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7dd5f5e0 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c0aaf640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd5dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61c1ab1640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd5eff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f61b40ccf50 tx=0x7f61b409fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e77ff717667aeaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.312+0000 7f61a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61a4ff9640 1 -- 172.21.15.31:0/3266748781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b401f040 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61a4ff9640 1 -- 172.21.15.31:0/3266748781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f61b40b4b40 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b40a45b0 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61a4ff9640 1 -- 172.21.15.31:0/3266748781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61b4099a90 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 10 client.15789 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb7ddcb3c0 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61a4ff9640 1 -- 172.21.15.31:0/3266748781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b40a45b0 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61a4ff9640 1 -- 172.21.15.31:0/3266748781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61b400f040 con 0x55bb7dd4b9b0 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 10 client.15789 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 20 client.15789 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7ddcbd10 0x55bb7ddce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 4 client.15789 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb7dcb1540 con 0x55bb7ddcbd10 2024-04-25T00:35:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.313+0000 7f61c12b0640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7ddcbd10 0x55bb7ddce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.314+0000 7f61c12b0640 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7ddcbd10 0x55bb7ddce130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa690a22229bb10 server_cookie=201eb676003fc45b in_seq=0 out_seq=0 2024-04-25T00:35:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.314+0000 7f61a4ff9640 10 client.15789 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61a4ff9640 1 -- 172.21.15.31:0/3266748781 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb7dcb1540 con 0x55bb7ddcbd10 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61a4ff9640 10 client.15789 handle_command_reply: tid=0 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 client.15789 shutdown 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 msgr2=0x55bb7dd5eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd5eff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f61b40ccf50 tx=0x7f61b409fe70 comp rx=0 tx=0).stop 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 shutdown_connections 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55bb7ddcbd10 0x55bb7ddce130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb7dd4b9b0 0x55bb7dd5eff0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb7dd46840 0x55bb7dd5dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 --2- 172.21.15.31:0/3266748781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb7dd45e50 0x55bb7dd5daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 >> 172.21.15.31:0/3266748781 conn(0x55bb7dd414c0 msgr2=0x55bb7dd58e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61a4ff9640 0 client.15789 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 shutdown_connections 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 1 -- 172.21.15.31:0/3266748781 wait complete. 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.331+0000 7f61c6224740 20 client.15789 trim_cache size 0 max 16384 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:35:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:35:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61a80a2cb0 msgr2=0x7f61a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61a80a2cb0 0x7f61a80a5170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e780 tx=0x7f61ac016000 comp rx=0 tx=0).stop 2024-04-25T00:35:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 msgr2=0x7f61bc1a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 0x7f61bc1a10e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f61b400eaa0 tx=0x7f61b400ef60 comp rx=0 tx=0).stop 2024-04-25T00:35:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 shutdown_connections 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f61a80a2cb0 0x7f61a80a5170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61bc105580 0x7f61bc1a10e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc104bc0 0x7f61bc19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 --2- 172.21.15.31:0/2189155871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61bc1039d0 0x7f61bc199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 >> 172.21.15.31:0/2189155871 conn(0x7f61bc0fef60 msgr2=0x7f61bc107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 shutdown_connections 2024-04-25T00:35:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:32.332+0000 7f61c3514640 1 -- 172.21.15.31:0/2189155871 wait complete. 2024-04-25T00:35:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:32 smithi169 ceph-mon[33112]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:32 smithi040 ceph-mon[32945]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:33.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:32 smithi031 ceph-mon[29167]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:34 smithi169 ceph-mon[33112]: pgmap v511: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:35:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:34 smithi040 ceph-mon[32945]: pgmap v511: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:35:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:34 smithi031 ceph-mon[29167]: pgmap v511: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:35:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:36 smithi169 ceph-mon[33112]: pgmap v512: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:36 smithi040 ceph-mon[32945]: pgmap v512: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:37.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:36 smithi031 ceph-mon[29167]: pgmap v512: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:38 smithi169 ceph-mon[33112]: pgmap v513: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:35:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:38 smithi040 ceph-mon[32945]: pgmap v513: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:35:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:38 smithi031 ceph-mon[29167]: pgmap v513: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:35:40.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:40 smithi169 ceph-mon[33112]: pgmap v514: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:40 smithi040 ceph-mon[32945]: pgmap v514: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:41.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:40 smithi031 ceph-mon[29167]: pgmap v514: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:35:42.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:42 smithi169 ceph-mon[33112]: pgmap v515: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:35:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:42 smithi040 ceph-mon[32945]: pgmap v515: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:35:43.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:42 smithi031 ceph-mon[29167]: pgmap v515: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:35:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:44 smithi169 ceph-mon[33112]: pgmap v516: 97 pgs: 97 active+clean; 3.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:35:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:44 smithi040 ceph-mon[32945]: pgmap v516: 97 pgs: 97 active+clean; 3.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:35:45.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:44 smithi031 ceph-mon[29167]: pgmap v516: 97 pgs: 97 active+clean; 3.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 -- 172.21.15.31:0/2738350587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481029b0 msgr2=0x7f3e48102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/2738350587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481029b0 0x7f3e48102e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009900 tx=0x7f3e3402a410 comp rx=0 tx=0).stop 2024-04-25T00:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 -- 172.21.15.31:0/2738350587 shutdown_connections 2024-04-25T00:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/2738350587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e48103370 0x7f3e48109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/2738350587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481029b0 0x7f3e48102e30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/2738350587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e481017c0 0x7f3e48101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 -- 172.21.15.31:0/2738350587 >> 172.21.15.31:0/2738350587 conn(0x7f3e480fcd30 msgr2=0x7f3e480ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 -- 172.21.15.31:0/2738350587 shutdown_connections 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.431+0000 7f3e4eca5640 1 -- 172.21.15.31:0/2738350587 wait complete. 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 Processor -- start 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 -- start start 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 0x7f3e480754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e481029b0 0x7f3e48073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e48103370 0x7f3e48074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810d2f0 con 0x7f3e481017c0 2024-04-25T00:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810d170 con 0x7f3e48103370 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4eca5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810d470 con 0x7f3e481029b0 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 0x7f3e480754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 0x7f3e480754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56486/0 (socket says 172.21.15.31:56486) 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 -- 172.21.15.31:0/1229673383 learned_addr learned my addr 172.21.15.31:0/1229673383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e47fff640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e481029b0 0x7f3e48073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4d242640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e48103370 0x7f3e48074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 -- 172.21.15.31:0/1229673383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e481029b0 msgr2=0x7f3e48073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e481029b0 0x7f3e48073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 -- 172.21.15.31:0/1229673383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e48103370 msgr2=0x7f3e48074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e48103370 0x7f3e48074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.432+0000 7f3e4ca41640 1 -- 172.21.15.31:0/1229673383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e48074900 con 0x7f3e481017c0 2024-04-25T00:35:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.433+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 0x7f3e480754f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00d950 tx=0x7f3e3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.433+0000 7f3e45ffb640 1 -- 172.21.15.31:0/1229673383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c017070 con 0x7f3e481017c0 2024-04-25T00:35:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.433+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481a1a50 con 0x7f3e481017c0 2024-04-25T00:35:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.433+0000 7f3e45ffb640 1 -- 172.21.15.31:0/1229673383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3c009c30 con 0x7f3e481017c0 2024-04-25T00:35:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.433+0000 7f3e45ffb640 1 -- 172.21.15.31:0/1229673383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00f620 con 0x7f3e481017c0 2024-04-25T00:35:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.433+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481a1f20 con 0x7f3e481017c0 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.436+0000 7f3e45ffb640 1 -- 172.21.15.31:0/1229673383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3e3c01a070 con 0x7f3e481017c0 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.436+0000 7f3e519b5740 1 Processor -- start 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.436+0000 7f3e519b5740 1 -- start start 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df83e50 0x56420df84270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df89340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df899b0 0x56420df8be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e519b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56420dee7cb0 con 0x56420df84840 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e519b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56420de905b0 con 0x56420df899b0 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e519b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56420dee3c70 con 0x56420df83e50 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df89340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df89340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56502/0 (socket says 172.21.15.31:56502) 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 -- 172.21.15.31:0/3332266648 learned_addr learned my addr 172.21.15.31:0/3332266648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e45ffb640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e380a2c30 0x7f3e380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4d242640 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df83e50 0x56420df84270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e47fff640 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df899b0 0x56420df8be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 -- 172.21.15.31:0/3332266648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df83e50 msgr2=0x56420df84270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df83e50 0x56420df84270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4d242640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e380a2c30 0x7f3e380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 -- 172.21.15.31:0/3332266648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df899b0 msgr2=0x56420df8be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df899b0 0x56420df8be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 -- 172.21.15.31:0/3332266648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420dbfc210 con 0x56420df84840 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e47fff640 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df899b0 0x56420df8be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e45ffb640 1 -- 172.21.15.31:0/1229673383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3e3c0cea70 con 0x7f3e481017c0 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.437+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df89340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c09aff0 tx=0x7f3e3c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1207a9ec222b365d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e4d242640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e380a2c30 0x7f3e380a50f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005c40 tx=0x7f3e40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e32ffd640 1 -- 172.21.15.31:0/3332266648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c095e60 con 0x56420df84840 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e32ffd640 1 -- 172.21.15.31:0/3332266648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3c009de0 con 0x56420df84840 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e32ffd640 1 -- 172.21.15.31:0/3332266648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0af420 con 0x56420df84840 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 -- 172.21.15.31:0/3332266648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 msgr2=0x56420df89340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df89340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c09aff0 tx=0x7f3e3c023000 comp rx=0 tx=0).stop 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 -- 172.21.15.31:0/3332266648 shutdown_connections 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df899b0 0x56420df8be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df89340 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 --2- 172.21.15.31:0/3332266648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df83e50 0x56420df84270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 -- 172.21.15.31:0/3332266648 >> 172.21.15.31:0/3332266648 conn(0x56420df7f4c0 msgr2=0x56420df7f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.438+0000 7f3e519b5740 1 -- 172.21.15.31:0/3332266648 shutdown_connections 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 -- 172.21.15.31:0/3332266648 wait complete. 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 -- start start 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 Processor -- start 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 0x56420df8e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df84840 0x56420df8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df95700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56420df8c8a0 con 0x56420df83e50 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56420dea3b50 con 0x56420df84840 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56420df8ca20 con 0x56420df899b0 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e519b5740 10 client.0 _ll_register_callbacks cb 0x56420dedeca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.439+0000 7f3e4ca41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 0x56420df8e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 0x56420df8e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56508/0 (socket says 172.21.15.31:56508) 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 -- 172.21.15.31:0/2972462605 learned_addr learned my addr 172.21.15.31:0/2972462605 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e47fff640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df84840 0x56420df8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4d242640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df95700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 -- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 msgr2=0x56420df95700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df95700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 -- 172.21.15.31:0/2972462605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df84840 msgr2=0x56420df8e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df84840 0x56420df8e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 -- 172.21.15.31:0/2972462605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56420dcc8620 con 0x56420df83e50 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 0x56420df8e180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c097cc0 tx=0x7f3e3c09b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2dca1b3fa93d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e27fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e27fff640 1 -- 172.21.15.31:0/2972462605 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0cee60 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e27fff640 1 -- 172.21.15.31:0/2972462605 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3e3c09b550 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c009de0 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e27fff640 1 -- 172.21.15.31:0/2972462605 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e3c095e60 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e519b5740 10 client.15801 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.440+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56420df8c8a0 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e27fff640 1 -- 172.21.15.31:0/2972462605 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3c009de0 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e27fff640 1 -- 172.21.15.31:0/2972462605 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3e3c09f1b0 con 0x56420df83e50 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e519b5740 10 client.15801 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e519b5740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e519b5740 20 client.15801 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420df99c90 0x56420df9c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e519b5740 4 client.15801 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56420def51f0 con 0x56420df99c90 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.441+0000 7f3e47fff640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420df99c90 0x56420df9c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.442+0000 7f3e27fff640 10 client.15801 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.442+0000 7f3e47fff640 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420df99c90 0x56420df9c080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fac53381666bc72 server_cookie=c3f6befae910bd68 in_seq=0 out_seq=0 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.442+0000 7f3e27fff640 1 -- 172.21.15.31:0/2972462605 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56420def51f0 con 0x56420df99c90 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.442+0000 7f3e27fff640 10 client.15801 handle_command_reply: tid=0 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.442+0000 7f3e519b5740 1 client.15801 shutdown 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 msgr2=0x56420df8e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 0x56420df8e180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c097cc0 tx=0x7f3e3c09b2f0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 shutdown_connections 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420df99c90 0x56420df9c080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df95700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df84840 0x56420df8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2972462605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df83e50 0x56420df8e180 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e27fff640 0 client.15801 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 >> 172.21.15.31:0/2972462605 conn(0x56420df7f4c0 msgr2=0x56420df7e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 shutdown_connections 2024-04-25T00:35:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 1 -- 172.21.15.31:0/2972462605 wait complete. 2024-04-25T00:35:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.443+0000 7f3e519b5740 20 client.15801 trim_cache size 0 max 16384 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 Processor -- start 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 -- start start 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df8cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df84840 0x56420df8d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 0x56420df7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481aae00 con 0x56420df899b0 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481ac940 con 0x56420df83e50 2024-04-25T00:35:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.458+0000 7f3e519b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481ab4d0 con 0x56420df84840 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 0x56420df7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 0x56420df7dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56522/0 (socket says 172.21.15.31:56522) 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 -- 172.21.15.31:0/2880401560 learned_addr learned my addr 172.21.15.31:0/2880401560 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e47fff640 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df84840 0x56420df8d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df8cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 -- 172.21.15.31:0/2880401560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df84840 msgr2=0x56420df8d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df84840 0x56420df8d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 -- 172.21.15.31:0/2880401560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 msgr2=0x56420df8cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df8cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 -- 172.21.15.31:0/2880401560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420df84c20 con 0x56420df899b0 2024-04-25T00:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e4d242640 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 0x56420df7dd30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3e40003d20 tx=0x7f3e40005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5478488477e2210b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e327fc640 1 -- 172.21.15.31:0/2880401560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400107c0 con 0x56420df899b0 2024-04-25T00:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e327fc640 1 -- 172.21.15.31:0/2880401560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e40024070 con 0x56420df899b0 2024-04-25T00:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.459+0000 7f3e327fc640 1 -- 172.21.15.31:0/2880401560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40020d40 con 0x56420df899b0 2024-04-25T00:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 -- 172.21.15.31:0/2880401560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 msgr2=0x56420df7dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 0x56420df7dd30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3e40003d20 tx=0x7f3e40005bd0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 -- 172.21.15.31:0/2880401560 shutdown_connections 2024-04-25T00:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df899b0 0x56420df7dd30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df84840 0x56420df8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 --2- 172.21.15.31:0/2880401560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df8cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 -- 172.21.15.31:0/2880401560 >> 172.21.15.31:0/2880401560 conn(0x56420df7f4c0 msgr2=0x56420df8dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 -- 172.21.15.31:0/2880401560 shutdown_connections 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 -- 172.21.15.31:0/2880401560 wait complete. 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 -- start start 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.460+0000 7f3e519b5740 1 Processor -- start 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df91df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df92330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df93230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481ac270 con 0x56420df84840 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481abba0 con 0x56420df83e50 2024-04-25T00:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481a8de0 con 0x56420df899b0 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e519b5740 10 client.0 _ll_register_callbacks cb 0x7f3e3c09aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df92330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df92330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56530/0 (socket says 172.21.15.31:56530) 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 -- 172.21.15.31:0/1085851650 learned_addr learned my addr 172.21.15.31:0/1085851650 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e4d242640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df93230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df91df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 -- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 msgr2=0x56420df93230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df93230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 -- 172.21.15.31:0/1085851650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 msgr2=0x56420df91df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df91df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 -- 172.21.15.31:0/1085851650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56420df93870 con 0x56420df84840 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e4d242640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df93230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df91df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e47fff640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df92330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006010 tx=0x7f3e34031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ca1704f4348e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.461+0000 7f3e27fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e27fff640 1 -- 172.21.15.31:0/1085851650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34035040 con 0x56420df84840 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e27fff640 1 -- 172.21.15.31:0/1085851650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3e34045dc0 con 0x56420df84840 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e40024070 con 0x56420df84840 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e27fff640 1 -- 172.21.15.31:0/1085851650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e34054050 con 0x56420df84840 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56420df94c10 con 0x56420df84840 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e27fff640 1 -- 172.21.15.31:0/1085851650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e40024070 con 0x56420df84840 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e27fff640 1 -- 172.21.15.31:0/1085851650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3e340035f0 con 0x56420df84840 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 20 client.15807 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420e000570 0x56420e002990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 4 client.15807 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.462+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56420deef540 con 0x56420e000570 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.463+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420e000570 0x56420e002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.463+0000 7f3e4ca41640 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420e000570 0x56420e002990 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf0cbbe21e0e20c server_cookie=91325334b9224b27 in_seq=0 out_seq=0 2024-04-25T00:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.463+0000 7f3e27fff640 10 client.15807 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e27fff640 1 -- 172.21.15.31:0/1085851650 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56420deef540 con 0x56420e000570 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e27fff640 10 client.15807 handle_command_reply: tid=0 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 client.15807 shutdown 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 msgr2=0x56420df92330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df92330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006010 tx=0x7f3e34031610 comp rx=0 tx=0).stop 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 shutdown_connections 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56420e000570 0x56420e002990 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56420df899b0 0x56420df93230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e27fff640 0 client.15807 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56420df84840 0x56420df92330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 --2- 172.21.15.31:0/1085851650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56420df83e50 0x56420df91df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 >> 172.21.15.31:0/1085851650 conn(0x56420df7f4c0 msgr2=0x56420df8cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 shutdown_connections 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 1 -- 172.21.15.31:0/1085851650 wait complete. 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.479+0000 7f3e519b5740 20 client.15807 trim_cache size 0 max 16384 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:35:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:35:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e380a2c30 msgr2=0x7f3e380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e380a2c30 0x7f3e380a50f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005c40 tx=0x7f3e40009500 comp rx=0 tx=0).stop 2024-04-25T00:35:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 msgr2=0x7f3e480754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 0x7f3e480754f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00d950 tx=0x7f3e3c00de10 comp rx=0 tx=0).stop 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 shutdown_connections 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e48103370 0x7f3e48074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e481029b0 0x7f3e48073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e380a2c30 0x7f3e380a50f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 --2- 172.21.15.31:0/1229673383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e481017c0 0x7f3e480754f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 >> 172.21.15.31:0/1229673383 conn(0x7f3e480fcd30 msgr2=0x7f3e48105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 shutdown_connections 2024-04-25T00:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:45.480+0000 7f3e4eca5640 1 -- 172.21.15.31:0/1229673383 wait complete. 2024-04-25T00:35:46.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:46 smithi169 ceph-mon[33112]: pgmap v517: 97 pgs: 97 active+clean; 3.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:35:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:46 smithi040 ceph-mon[32945]: pgmap v517: 97 pgs: 97 active+clean; 3.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:35:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:46 smithi031 ceph-mon[29167]: pgmap v517: 97 pgs: 97 active+clean; 3.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:35:48.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:48 smithi169 ceph-mon[33112]: pgmap v518: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:35:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:48 smithi040 ceph-mon[32945]: pgmap v518: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:35:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:48 smithi031 ceph-mon[29167]: pgmap v518: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:35:50.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:50 smithi169 ceph-mon[33112]: pgmap v519: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:35:50.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:50 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:35:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:50 smithi040 ceph-mon[32945]: pgmap v519: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:35:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:50 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:35:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:50 smithi031 ceph-mon[29167]: pgmap v519: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:35:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:50 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:35:51.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:35:51.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:35:51.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:35:51.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:35:51.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:35:51.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:35:52.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:35:52.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:35:52.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:35:53.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:52 smithi169 ceph-mon[33112]: pgmap v520: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:52 smithi040 ceph-mon[32945]: pgmap v520: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:52 smithi031 ceph-mon[29167]: pgmap v520: 97 pgs: 97 active+clean; 3.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:55.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:54 smithi040 ceph-mon[32945]: pgmap v521: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:55.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:54 smithi031 ceph-mon[29167]: pgmap v521: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:54 smithi169 ceph-mon[33112]: pgmap v521: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:35:57.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:56 smithi040 ceph-mon[32945]: pgmap v522: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:57.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:56 smithi169 ceph-mon[33112]: pgmap v522: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:56 smithi031 ceph-mon[29167]: pgmap v522: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:35:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 -- 172.21.15.31:0/2408107719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 msgr2=0x7fce84102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 --2- 172.21.15.31:0/2408107719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84102e40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fce74009900 tx=0x7fce7402a410 comp rx=0 tx=0).stop 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 -- 172.21.15.31:0/2408107719 shutdown_connections 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 --2- 172.21.15.31:0/2408107719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce84103380 0x7fce84109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 --2- 172.21.15.31:0/2408107719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84102e40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 --2- 172.21.15.31:0/2408107719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841017d0 0x7fce84101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 -- 172.21.15.31:0/2408107719 >> 172.21.15.31:0/2408107719 conn(0x7fce840fcd60 msgr2=0x7fce840ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 -- 172.21.15.31:0/2408107719 shutdown_connections 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.579+0000 7fce8ae83640 1 -- 172.21.15.31:0/2408107719 wait complete. 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 Processor -- start 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 -- start start 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce841017d0 0x7fce84197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce84103380 0x7fce8419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410d300 con 0x7fce841029c0 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410d180 con 0x7fce841017d0 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce8ae83640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410d480 con 0x7fce84103380 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54936/0 (socket says 172.21.15.31:54936) 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 -- 172.21.15.31:0/3534643981 learned_addr learned my addr 172.21.15.31:0/3534643981 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce88c1f640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce841017d0 0x7fce84197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce89420640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce84103380 0x7fce8419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 -- 172.21.15.31:0/3534643981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce84103380 msgr2=0x7fce8419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce84103380 0x7fce8419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 -- 172.21.15.31:0/3534643981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce841017d0 msgr2=0x7fce84197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce841017d0 0x7fce84197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce7bfff640 1 -- 172.21.15.31:0/3534643981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8419f590 con 0x7fce841029c0 2024-04-25T00:35:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.580+0000 7fce89420640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce84103380 0x7fce8419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.581+0000 7fce7bfff640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84197eb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce74004770 tx=0x7fce740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.581+0000 7fce79ffb640 1 -- 172.21.15.31:0/3534643981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7403f030 con 0x7fce841029c0 2024-04-25T00:35:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.581+0000 7fce79ffb640 1 -- 172.21.15.31:0/3534643981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce74003cb0 con 0x7fce841029c0 2024-04-25T00:35:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.581+0000 7fce79ffb640 1 -- 172.21.15.31:0/3534643981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74035070 con 0x7fce841029c0 2024-04-25T00:35:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.581+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8419f820 con 0x7fce841029c0 2024-04-25T00:35:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.581+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8419fd10 con 0x7fce841029c0 2024-04-25T00:35:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.583+0000 7fce79ffb640 1 -- 172.21.15.31:0/3534643981 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fce74042030 con 0x7fce841029c0 2024-04-25T00:35:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:35:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 -- start start 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac726680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac72b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac72e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564aac685cb0 con 0x564aac726260 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564aac62e5b0 con 0x564aac726c50 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce8db8b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564aac681c70 con 0x564aac72bdc0 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce79ffb640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce6c0a2c30 0x7fce6c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce89420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac726680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce89420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac726680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54952/0 (socket says 172.21.15.31:54952) 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce89420640 1 -- 172.21.15.31:0/3160664715 learned_addr learned my addr 172.21.15.31:0/3160664715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce88c1f640 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac72b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce89420640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce6c0a2c30 0x7fce6c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce7bfff640 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac72e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.584+0000 7fce79ffb640 1 -- 172.21.15.31:0/3534643981 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fce740efab0 con 0x7fce841029c0 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 -- 172.21.15.31:0/3160664715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 msgr2=0x564aac72e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac72e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 -- 172.21.15.31:0/3160664715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 msgr2=0x564aac72b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac72b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 -- 172.21.15.31:0/3160664715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aac39a210 con 0x564aac726260 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce88c1f640 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac72b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac726680 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fce7c012da0 tx=0x7fce7c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f552c33a465846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce89420640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce6c0a2c30 0x7fce6c0a50f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00afd0 tx=0x7fce7c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce6affd640 1 -- 172.21.15.31:0/3160664715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c019cb0 con 0x564aac726260 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce6affd640 1 -- 172.21.15.31:0/3160664715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c019e50 con 0x564aac726260 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.585+0000 7fce6affd640 1 -- 172.21.15.31:0/3160664715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c01c670 con 0x564aac726260 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 -- 172.21.15.31:0/3160664715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 msgr2=0x564aac726680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac726680 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fce7c012da0 tx=0x7fce7c00f730 comp rx=0 tx=0).stop 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 -- 172.21.15.31:0/3160664715 shutdown_connections 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac72e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac72b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3160664715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac726680 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 -- 172.21.15.31:0/3160664715 >> 172.21.15.31:0/3160664715 conn(0x564aac71d4c0 msgr2=0x564aac71d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 -- 172.21.15.31:0/3160664715 shutdown_connections 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 -- 172.21.15.31:0/3160664715 wait complete. 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 -- start start 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.586+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac7306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 0x564aac730bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564aac72ed10 con 0x564aac726c50 2024-04-25T00:35:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564aac641b50 con 0x564aac726260 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564aac72ee90 con 0x564aac72bdc0 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce8db8b740 10 client.0 _ll_register_callbacks cb 0x564aac67cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 0x564aac730bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 0x564aac730bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54966/0 (socket says 172.21.15.31:54966) 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 -- 172.21.15.31:0/2230828817 learned_addr learned my addr 172.21.15.31:0/2230828817 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce88c1f640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac7306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce89420640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 -- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 msgr2=0x564aac737c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 -- 172.21.15.31:0/2230828817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 msgr2=0x564aac7306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac7306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce7bfff640 1 -- 172.21.15.31:0/2230828817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aac466620 con 0x564aac726c50 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.587+0000 7fce89420640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce7bfff640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 0x564aac730bf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fce740bbf80 tx=0x7fce740bbe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da025ee63b558aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce63fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce63fff640 1 -- 172.21.15.31:0/2230828817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74049040 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce63fff640 1 -- 172.21.15.31:0/2230828817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fce740b88f0 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7c019e50 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce63fff640 1 -- 172.21.15.31:0/2230828817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce740c0630 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 10 client.15819 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564aac73afa0 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce63fff640 1 -- 172.21.15.31:0/2230828817 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c019e50 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce63fff640 1 -- 172.21.15.31:0/2230828817 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fce7403b460 con 0x564aac726c50 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 10 client.15819 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 20 client.15819 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac73c750 0x564aac73eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 4 client.15819 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.588+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aac6931f0 con 0x564aac73c750 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.589+0000 7fce88c1f640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac73c750 0x564aac73eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.589+0000 7fce88c1f640 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac73c750 0x564aac73eb90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462d1cfd5ee126b6 server_cookie=6dc98ed6e05d0802 in_seq=0 out_seq=0 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.589+0000 7fce63fff640 10 client.15819 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce63fff640 1 -- 172.21.15.31:0/2230828817 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564aac6931f0 con 0x564aac73c750 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce63fff640 10 client.15819 handle_command_reply: tid=0 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 client.15819 shutdown 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 msgr2=0x564aac730bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 0x564aac730bf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fce740bbf80 tx=0x7fce740bbe80 comp rx=0 tx=0).stop 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 shutdown_connections 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac73c750 0x564aac73eb90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726c50 0x564aac730bf0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce63fff640 0 client.15819 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 --2- 172.21.15.31:0/2230828817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac7306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 >> 172.21.15.31:0/2230828817 conn(0x564aac71d4c0 msgr2=0x564aac71c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 shutdown_connections 2024-04-25T00:35:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 1 -- 172.21.15.31:0/2230828817 wait complete. 2024-04-25T00:35:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.590+0000 7fce8db8b740 20 client.15819 trim_cache size 0 max 16384 2024-04-25T00:35:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.605+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:35:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.605+0000 7fce8db8b740 1 -- start start 2024-04-25T00:35:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac7280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac7285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce8db8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a6910 con 0x564aac726260 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce8db8b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a8450 con 0x564aac726c50 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce8db8b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a6fe0 con 0x564aac72bdc0 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac7280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac7280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54980/0 (socket says 172.21.15.31:54980) 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 -- 172.21.15.31:0/3692794143 learned_addr learned my addr 172.21.15.31:0/3692794143 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce89420640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce7bfff640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac7285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 -- 172.21.15.31:0/3692794143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 msgr2=0x564aac737bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 -- 172.21.15.31:0/3692794143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 msgr2=0x564aac7285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac7285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 -- 172.21.15.31:0/3692794143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aac73f1c0 con 0x564aac726260 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce7bfff640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac7285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce89420640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce88c1f640 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac7280b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fce8000bdb0 tx=0x7fce8000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869dad89999ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce6a7fc640 1 -- 172.21.15.31:0/3692794143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80005220 con 0x564aac726260 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce6a7fc640 1 -- 172.21.15.31:0/3692794143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce80023030 con 0x564aac726260 2024-04-25T00:35:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.606+0000 7fce6a7fc640 1 -- 172.21.15.31:0/3692794143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8001f410 con 0x564aac726260 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 -- 172.21.15.31:0/3692794143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 msgr2=0x564aac7280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac7280b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fce8000bdb0 tx=0x7fce8000b390 comp rx=0 tx=0).stop 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 -- 172.21.15.31:0/3692794143 shutdown_connections 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac72bdc0 0x564aac737bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726c50 0x564aac7285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 --2- 172.21.15.31:0/3692794143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac726260 0x564aac7280b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 -- 172.21.15.31:0/3692794143 >> 172.21.15.31:0/3692794143 conn(0x564aac71d4c0 msgr2=0x564aac71d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 -- 172.21.15.31:0/3692794143 shutdown_connections 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 -- 172.21.15.31:0/3692794143 wait complete. 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 -- start start 2024-04-25T00:35:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.607+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac735280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac726c50 0x564aac7357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 0x564aac736710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a7d80 con 0x564aac72bdc0 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a76b0 con 0x564aac726260 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a48f0 con 0x564aac726c50 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce8db8b740 10 client.0 _ll_register_callbacks cb 0x7fce74009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 0x564aac736710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 0x564aac736710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54994/0 (socket says 172.21.15.31:54994) 2024-04-25T00:35:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 -- 172.21.15.31:0/427403621 learned_addr learned my addr 172.21.15.31:0/427403621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce88c1f640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac735280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce7bfff640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac726c50 0x564aac7357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 -- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac726c50 msgr2=0x564aac7357c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac726c50 0x564aac7357c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 -- 172.21.15.31:0/427403621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 msgr2=0x564aac735280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac735280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 -- 172.21.15.31:0/427403621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aac736d00 con 0x564aac72bdc0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce89420640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 0x564aac736710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x564aac7274b0 tx=0x7fce7c004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6a2059b13c273c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.608+0000 7fce63fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce63fff640 1 -- 172.21.15.31:0/427403621 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c033020 con 0x564aac72bdc0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce63fff640 1 -- 172.21.15.31:0/427403621 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fce7c02f450 con 0x564aac72bdc0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce80023030 con 0x564aac72bdc0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce63fff640 1 -- 172.21.15.31:0/427403621 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce7c03e050 con 0x564aac72bdc0 2024-04-25T00:35:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564aac737f80 con 0x564aac72bdc0 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce63fff640 1 -- 172.21.15.31:0/427403621 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce80023030 con 0x564aac72bdc0 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce63fff640 1 -- 172.21.15.31:0/427403621 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fce7c01b050 con 0x564aac72bdc0 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 20 client.15825 populate_metadata read hostname 'smithi031' 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac7a3280 0x564aac7a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 4 client.15825 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.609+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aac68d540 con 0x564aac7a3280 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.610+0000 7fce88c1f640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac7a3280 0x564aac7a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.610+0000 7fce63fff640 10 client.15825 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.610+0000 7fce88c1f640 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac7a3280 0x564aac7a56a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a2d59ef4647195 server_cookie=7c436de8ecbdc80f in_seq=0 out_seq=0 2024-04-25T00:35:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce63fff640 1 -- 172.21.15.31:0/427403621 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564aac68d540 con 0x564aac7a3280 2024-04-25T00:35:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce63fff640 10 client.15825 handle_command_reply: tid=0 2024-04-25T00:35:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 client.15825 shutdown 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 msgr2=0x564aac736710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 0x564aac736710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x564aac7274b0 tx=0x7fce7c004a70 comp rx=0 tx=0).stop 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 shutdown_connections 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564aac7a3280 0x564aac7a56a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564aac72bdc0 0x564aac736710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564aac726c50 0x564aac7357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 --2- 172.21.15.31:0/427403621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564aac726260 0x564aac735280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce63fff640 0 client.15825 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.625+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 >> 172.21.15.31:0/427403621 conn(0x564aac71d4c0 msgr2=0x564aac730650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 shutdown_connections 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8db8b740 1 -- 172.21.15.31:0/427403621 wait complete. 2024-04-25T00:35:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8db8b740 20 client.15825 trim_cache size 0 max 16384 2024-04-25T00:35:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:35:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:35:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:35:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce6c0a2c30 msgr2=0x7fce6c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce6c0a2c30 0x7fce6c0a50f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00afd0 tx=0x7fce7c01a000 comp rx=0 tx=0).stop 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 msgr2=0x7fce84197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84197eb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce74004770 tx=0x7fce740042f0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 shutdown_connections 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce84103380 0x7fce8419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fce6c0a2c30 0x7fce6c0a50f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce841029c0 0x7fce84197eb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 --2- 172.21.15.31:0/3534643981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce841017d0 0x7fce84197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.626+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 >> 172.21.15.31:0/3534643981 conn(0x7fce840fcd60 msgr2=0x7fce84105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.627+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 shutdown_connections 2024-04-25T00:35:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:35:58.627+0000 7fce8ae83640 1 -- 172.21.15.31:0/3534643981 wait complete. 2024-04-25T00:35:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:35:58 smithi031 ceph-mon[29167]: pgmap v523: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:59.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:35:58 smithi169 ceph-mon[33112]: pgmap v523: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:35:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:35:58 smithi040 ceph-mon[32945]: pgmap v523: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:00.307 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.307 INFO:tasks.workunit.client.0.smithi031.stdout:189/600: open fd = 3 2024-04-25T00:36:00.307 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:190/600: open fd = 3 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:191/600: open fd = 3 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:192/600: open fd = 3 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.308 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:193/600: open fd = 3 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:194/600: open fd = 3 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.309 INFO:tasks.workunit.client.0.smithi031.stdout:195/600: open fd = 3 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:196/600: open fd = 3 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:197/600: open fd = 3 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.310 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:198/600: open fd = 3 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:199/600: open fd = 3 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:200/600: open fd = 3 2024-04-25T00:36:00.311 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:201/600: open fd = 3 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:202/600: open fd = 3 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.312 INFO:tasks.workunit.client.0.smithi031.stdout:203/600: open fd = 3 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:204/600: open fd = 3 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:205/600: open fd = 3 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:206/600: open fd = 3 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.313 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:207/600: open fd = 3 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:208/600: open fd = 3 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:209/600: open fd = 3 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:210/600: open fd = 3 2024-04-25T00:36:00.314 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:211/600: open fd = 3 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:212/600: open fd = 3 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:213/600: open fd = 3 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.315 INFO:tasks.workunit.client.0.smithi031.stdout:214/600: open fd = 3 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:215/600: open fd = 3 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:216/600: open fd = 3 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.316 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:217/600: open fd = 3 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:218/600: open fd = 3 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:219/600: open fd = 3 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:220/600: open fd = 3 2024-04-25T00:36:00.317 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:221/600: open fd = 3 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:222/600: open fd = 3 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:223/600: open fd = 3 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.318 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:224/600: open fd = 3 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:225/600: open fd = 3 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:226/600: open fd = 3 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:227/600: open fd = 3 2024-04-25T00:36:00.319 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:228/600: open fd = 3 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:229/600: open fd = 3 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:230/600: open fd = 3 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.320 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:231/600: open fd = 3 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:232/600: open fd = 3 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:233/600: open fd = 3 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.321 INFO:tasks.workunit.client.0.smithi031.stdout:234/600: open fd = 3 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:235/600: open fd = 3 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:236/600: open fd = 3 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:237/600: open fd = 3 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.322 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:238/600: open fd = 3 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:239/600: open fd = 3 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:240/600: open fd = 3 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:241/600: open fd = 3 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.323 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:242/600: open fd = 3 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:243/600: open fd = 3 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:244/600: open fd = 3 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:245/600: open fd = 3 2024-04-25T00:36:00.324 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:246/600: open fd = 3 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:247/600: open fd = 3 2024-04-25T00:36:00.325 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:248/600: open fd = 3 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:249/600: open fd = 3 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:250/600: open fd = 3 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:00.326 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:36:01.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:00 smithi031 ceph-mon[29167]: pgmap v524: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:01.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:00 smithi169 ceph-mon[33112]: pgmap v524: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:00 smithi040 ceph-mon[32945]: pgmap v524: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:02 smithi031 ceph-mon[29167]: pgmap v525: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:36:03.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:02 smithi169 ceph-mon[33112]: pgmap v525: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:36:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:02 smithi040 ceph-mon[32945]: pgmap v525: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:36:05.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:04 smithi031 ceph-mon[29167]: pgmap v526: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:05.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:04 smithi169 ceph-mon[33112]: pgmap v526: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:05.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:04 smithi040 ceph-mon[32945]: pgmap v526: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:06 smithi031 ceph-mon[29167]: pgmap v527: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:07.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:06 smithi169 ceph-mon[33112]: pgmap v527: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:06 smithi040 ceph-mon[32945]: pgmap v527: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:08 smithi031 ceph-mon[29167]: pgmap v528: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:09.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:08 smithi169 ceph-mon[33112]: pgmap v528: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:08 smithi040 ceph-mon[32945]: pgmap v528: 97 pgs: 97 active+clean; 3.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:10 smithi031 ceph-mon[29167]: pgmap v529: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:11.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:10 smithi169 ceph-mon[33112]: pgmap v529: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:10 smithi040 ceph-mon[32945]: pgmap v529: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 -- 172.21.15.31:0/1597986130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c0103380 msgr2=0x7fd7c0109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 --2- 172.21.15.31:0/1597986130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c0103380 0x7fd7c0109c10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009ca0 tx=0x7fd7b802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 -- 172.21.15.31:0/1597986130 shutdown_connections 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 --2- 172.21.15.31:0/1597986130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c0103380 0x7fd7c0109c10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 --2- 172.21.15.31:0/1597986130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c01029c0 0x7fd7c0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 --2- 172.21.15.31:0/1597986130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01017d0 0x7fd7c0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 -- 172.21.15.31:0/1597986130 >> 172.21.15.31:0/1597986130 conn(0x7fd7c00fcd60 msgr2=0x7fd7c00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 -- 172.21.15.31:0/1597986130 shutdown_connections 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.725+0000 7fd7c6431640 1 -- 172.21.15.31:0/1597986130 wait complete. 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 Processor -- start 2024-04-25T00:36:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 -- start start 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 0x7fd7c0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01029c0 0x7fd7c019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c0103380 0x7fd7c01a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010d300 con 0x7fd7c01017d0 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010d180 con 0x7fd7c0103380 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7c6431640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010d480 con 0x7fd7c01029c0 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.726+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 0x7fd7c0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 0x7fd7c0199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52154/0 (socket says 172.21.15.31:52154) 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 -- 172.21.15.31:0/2385067521 learned_addr learned my addr 172.21.15.31:0/2385067521 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c0103380 0x7fd7c01a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01029c0 0x7fd7c019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 -- 172.21.15.31:0/2385067521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01029c0 msgr2=0x7fd7c019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01029c0 0x7fd7c019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 -- 172.21.15.31:0/2385067521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c0103380 msgr2=0x7fd7c01a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c0103380 0x7fd7c01a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 -- 172.21.15.31:0/2385067521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c01a1700 con 0x7fd7c01017d0 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01029c0 0x7fd7c019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bffff640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 0x7fd7c0199ae0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000d950 tx=0x7fd7b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bd7fa640 1 -- 172.21.15.31:0/2385067521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0017070 con 0x7fd7c01017d0 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01a19f0 con 0x7fd7c01017d0 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bd7fa640 1 -- 172.21.15.31:0/2385067521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7b0009c30 con 0x7fd7c01017d0 2024-04-25T00:36:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7bd7fa640 1 -- 172.21.15.31:0/2385067521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000f620 con 0x7fd7c01017d0 2024-04-25T00:36:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.727+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01a1ef0 con 0x7fd7c01017d0 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7bd7fa640 1 -- 172.21.15.31:0/2385067521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd7b001a070 con 0x7fd7c01017d0 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 Processor -- start 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 -- start start 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307bae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55da30716cb0 con 0x55da307b3840 2024-04-25T00:36:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55da306bf5b0 con 0x55da307b2e50 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c921c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55da30712c70 con 0x55da307b89b0 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52158/0 (socket says 172.21.15.31:52158) 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7bffff640 1 -- 172.21.15.31:0/2755854248 learned_addr learned my addr 172.21.15.31:0/2755854248 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.730+0000 7fd7bd7fa640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd7a80a2c30 0x7fd7a80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd7a80a2c30 0x7fd7a80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bffff640 1 -- 172.21.15.31:0/2755854248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 msgr2=0x55da307bae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bffff640 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307bae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bffff640 1 -- 172.21.15.31:0/2755854248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 msgr2=0x55da307b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bffff640 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bd7fa640 1 -- 172.21.15.31:0/2385067521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd7b00cead0 con 0x7fd7c01017d0 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bffff640 1 -- 172.21.15.31:0/2755854248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3042b210 con 0x55da307b3840 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7bffff640 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307b8340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd7b009b290 tx=0x7fd7b0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381fbbdff650ab9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd7a80a2c30 0x7fd7a80a50f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8004af0 tx=0x7fd7b802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7a67fc640 1 -- 172.21.15.31:0/2755854248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0021070 con 0x55da307b3840 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7a67fc640 1 -- 172.21.15.31:0/2755854248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7b009ba00 con 0x55da307b3840 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.731+0000 7fd7a67fc640 1 -- 172.21.15.31:0/2755854248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00d2100 con 0x55da307b3840 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 -- 172.21.15.31:0/2755854248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 msgr2=0x55da307b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307b8340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd7b009b290 tx=0x7fd7b0005f30 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 -- 172.21.15.31:0/2755854248 shutdown_connections 2024-04-25T00:36:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307bae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307b8340 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 --2- 172.21.15.31:0/2755854248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 -- 172.21.15.31:0/2755854248 >> 172.21.15.31:0/2755854248 conn(0x55da307ae4c0 msgr2=0x55da307ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 -- 172.21.15.31:0/2755854248 shutdown_connections 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 -- 172.21.15.31:0/2755854248 wait complete. 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.732+0000 7fd7c921c740 1 -- start start 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 Processor -- start 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b3840 0x55da307bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 0x55da307c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55da307bb8a0 con 0x55da307b89b0 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55da306d2b50 con 0x55da307b2e50 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55da307bba20 con 0x55da307b3840 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c921c740 10 client.0 _ll_register_callbacks cb 0x55da3070dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 0x55da307c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 0x55da307c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52166/0 (socket says 172.21.15.31:52166) 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 -- 172.21.15.31:0/1537115198 learned_addr learned my addr 172.21.15.31:0/1537115198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b3840 0x55da307bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 -- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b3840 msgr2=0x55da307bd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b3840 0x55da307bd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 -- 172.21.15.31:0/1537115198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 msgr2=0x55da307bd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307bd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.733+0000 7fd7c49ce640 1 -- 172.21.15.31:0/1537115198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da304f7620 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 0x55da307c4680 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8035000 tx=0x7fd7b80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad534f063b87a8b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd79f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1537115198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80045d0 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1537115198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd7b8004770 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b009ba00 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1537115198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7b80305c0 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 10 client.15837 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da307c79f0 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1537115198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7b009ba00 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1537115198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd7b802f7f0 con 0x55da307b89b0 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 10 client.15837 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 20 client.15837 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da307c9130 0x55da307cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 4 client.15837 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.734+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da307241f0 con 0x55da307c9130 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.735+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da307c9130 0x55da307cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.735+0000 7fd79f7fe640 10 client.15837 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.735+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da307c9130 0x55da307cb5c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd60c3c2c2293568 server_cookie=e43fc93e1e6ea22f in_seq=0 out_seq=0 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1537115198 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da307241f0 con 0x55da307c9130 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd79f7fe640 10 client.15837 handle_command_reply: tid=0 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 client.15837 shutdown 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 msgr2=0x55da307c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 0x55da307c4680 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8035000 tx=0x7fd7b80095c0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 shutdown_connections 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da307c9130 0x55da307cb5c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b89b0 0x55da307c4680 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b3840 0x55da307bd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1537115198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd79f7fe640 0 client.15837 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 >> 172.21.15.31:0/1537115198 conn(0x55da307ae4c0 msgr2=0x55da307ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 shutdown_connections 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.736+0000 7fd7c921c740 1 -- 172.21.15.31:0/1537115198 wait complete. 2024-04-25T00:36:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.737+0000 7fd7c921c740 20 client.15837 trim_cache size 0 max 16384 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.751+0000 7fd7c921c740 1 Processor -- start 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 -- start start 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01aae00 con 0x55da307b3840 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01ac940 con 0x55da307b2e50 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c921c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01ab4d0 con 0x55da307b89b0 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52180/0 (socket says 172.21.15.31:52180) 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/266317500 learned_addr learned my addr 172.21.15.31:0/266317500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bffff640 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/266317500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 msgr2=0x55da307ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/266317500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 msgr2=0x55da307acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/266317500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da307ae060 con 0x55da307b3840 2024-04-25T00:36:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c4640 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400bd90 tx=0x7fd7b4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed983a4e13da8a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7a5ffb640 1 -- 172.21.15.31:0/266317500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40040f0 con 0x55da307b3840 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.752+0000 7fd7a5ffb640 1 -- 172.21.15.31:0/266317500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7b4004290 con 0x55da307b3840 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7a5ffb640 1 -- 172.21.15.31:0/266317500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4010830 con 0x55da307b3840 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 -- 172.21.15.31:0/266317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 msgr2=0x55da307c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c4640 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400bd90 tx=0x7fd7b4012820 comp rx=0 tx=0).stop 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 -- 172.21.15.31:0/266317500 shutdown_connections 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c4640 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 --2- 172.21.15.31:0/266317500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 -- 172.21.15.31:0/266317500 >> 172.21.15.31:0/266317500 conn(0x55da307ae4c0 msgr2=0x55da307b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 -- 172.21.15.31:0/266317500 shutdown_connections 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 -- 172.21.15.31:0/266317500 wait complete. 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.753+0000 7fd7c921c740 1 -- start start 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 Processor -- start 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01ac270 con 0x55da307b3840 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01abba0 con 0x55da307b2e50 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01a8de0 con 0x55da307b89b0 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c921c740 10 client.0 _ll_register_callbacks cb 0x7fd7b8009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52192/0 (socket says 172.21.15.31:52192) 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/1370245840 learned_addr learned my addr 172.21.15.31:0/1370245840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 msgr2=0x55da307c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/1370245840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 msgr2=0x55da307c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bf7fe640 1 -- 172.21.15.31:0/1370245840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da307c3ee0 con 0x55da307b3840 2024-04-25T00:36:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7c49ce640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.754+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7bf7fe640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c28e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400bbd0 tx=0x7fd7b400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7493b23846e62da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd79f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1370245840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4021040 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1370245840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd7b4005100 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b4004290 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1370245840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7b4005300 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 10 client.15843 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da3082fde0 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1370245840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7b4004290 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1370245840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd7b4024070 con 0x55da307b3840 2024-04-25T00:36:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 20 client.15843 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da30830750 0x55da30832b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 4 client.15843 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.755+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da3071e540 con 0x55da30830750 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.756+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da30830750 0x55da30832b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.756+0000 7fd7bffff640 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da30830750 0x55da30832b70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e58544a98ab2fdf server_cookie=f102d28e9c93920b in_seq=0 out_seq=0 2024-04-25T00:36:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.756+0000 7fd79f7fe640 10 client.15843 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.771+0000 7fd79f7fe640 1 -- 172.21.15.31:0/1370245840 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da3071e540 con 0x55da30830750 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.771+0000 7fd79f7fe640 10 client.15843 handle_command_reply: tid=0 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 client.15843 shutdown 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 msgr2=0x55da307c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c28e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400bbd0 tx=0x7fd7b400b920 comp rx=0 tx=0).stop 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 shutdown_connections 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55da30830750 0x55da30832b70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da307b89b0 0x55da307c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da307b3840 0x55da307c28e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 --2- 172.21.15.31:0/1370245840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da307b2e50 0x55da307c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd79f7fe640 0 client.15843 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 >> 172.21.15.31:0/1370245840 conn(0x55da307ae4c0 msgr2=0x55da307b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 shutdown_connections 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 1 -- 172.21.15.31:0/1370245840 wait complete. 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c921c740 20 client.15843 trim_cache size 0 max 16384 2024-04-25T00:36:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:36:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:36:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:36:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:36:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd7a80a2c30 msgr2=0x7fd7a80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c6431640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd7a80a2c30 0x7fd7a80a50f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8004af0 tx=0x7fd7b802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.772+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 msgr2=0x7fd7c0199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 0x7fd7c0199ae0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000d950 tx=0x7fd7b000de10 comp rx=0 tx=0).stop 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 shutdown_connections 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c0103380 0x7fd7c01a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c01029c0 0x7fd7c019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd7a80a2c30 0x7fd7a80a50f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 --2- 172.21.15.31:0/2385067521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c01017d0 0x7fd7c0199ae0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 >> 172.21.15.31:0/2385067521 conn(0x7fd7c00fcd60 msgr2=0x7fd7c0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 shutdown_connections 2024-04-25T00:36:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:11.773+0000 7fd7c6431640 1 -- 172.21.15.31:0/2385067521 wait complete. 2024-04-25T00:36:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:12 smithi031 ceph-mon[29167]: pgmap v530: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:12 smithi169 ceph-mon[33112]: pgmap v530: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:12 smithi040 ceph-mon[32945]: pgmap v530: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:14 smithi031 ceph-mon[29167]: pgmap v531: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:36:15.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:14 smithi169 ceph-mon[33112]: pgmap v531: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:36:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:14 smithi040 ceph-mon[32945]: pgmap v531: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:36:17.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:16 smithi031 ceph-mon[29167]: pgmap v532: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:16 smithi169 ceph-mon[33112]: pgmap v532: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:17.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:16 smithi040 ceph-mon[32945]: pgmap v532: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:19.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:18 smithi031 ceph-mon[29167]: pgmap v533: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:19.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:18 smithi169 ceph-mon[33112]: pgmap v533: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:19.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:18 smithi040 ceph-mon[32945]: pgmap v533: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:21.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:20 smithi031 ceph-mon[29167]: pgmap v534: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:21.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:20 smithi169 ceph-mon[33112]: pgmap v534: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:21.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:20 smithi040 ceph-mon[32945]: pgmap v534: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:22 smithi031 ceph-mon[29167]: pgmap v535: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:23.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:22 smithi169 ceph-mon[33112]: pgmap v535: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:23.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:22 smithi040 ceph-mon[32945]: pgmap v535: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/2968144679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 msgr2=0x7ff7c8108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/2968144679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8108bb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009cd0 tx=0x7ff7c002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/2968144679 shutdown_connections 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/2968144679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8108bb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/2968144679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c8100950 0x7ff7c8100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/2968144679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8101cb0 0x7ff7c81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/2968144679 >> 172.21.15.31:0/2968144679 conn(0x7ff7c80fb3c0 msgr2=0x7ff7c80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/2968144679 shutdown_connections 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/2968144679 wait complete. 2024-04-25T00:36:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 Processor -- start 2024-04-25T00:36:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.872+0000 7ff7cdd7e640 1 -- start start 2024-04-25T00:36:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7cdd7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8100950 0x7ff7c8197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7cdd7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7cdd7e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c8101cb0 0x7ff7c819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7cdd7e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c810c220 con 0x7ff7c8101310 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7cdd7e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c810c0a0 con 0x7ff7c8100950 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7cdd7e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c810c3a0 con 0x7ff7c8101cb0 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7c77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8100950 0x7ff7c8197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54074/0 (socket says 172.21.15.31:54074) 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 -- 172.21.15.31:0/350444950 learned_addr learned my addr 172.21.15.31:0/350444950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c8101cb0 0x7ff7c819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 -- 172.21.15.31:0/350444950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c8101cb0 msgr2=0x7ff7c819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c8101cb0 0x7ff7c819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 -- 172.21.15.31:0/350444950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8100950 msgr2=0x7ff7c8197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8100950 0x7ff7c8197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7beffd640 1 -- 172.21.15.31:0/350444950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c819f490 con 0x7ff7c8101310 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.873+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8100950 0x7ff7c8197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.874+0000 7ff7beffd640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8197db0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000e9f0 tx=0x7ff7b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.874+0000 7ff7c57fa640 1 -- 172.21.15.31:0/350444950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000cd80 con 0x7ff7c8101310 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.874+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c819f780 con 0x7ff7c8101310 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.874+0000 7ff7c57fa640 1 -- 172.21.15.31:0/350444950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7b00054d0 con 0x7ff7c8101310 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.874+0000 7ff7c57fa640 1 -- 172.21.15.31:0/350444950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0010490 con 0x7ff7c8101310 2024-04-25T00:36:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.874+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c819fcd0 con 0x7ff7c8101310 2024-04-25T00:36:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 Processor -- start 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7c57fa640 1 -- 172.21.15.31:0/350444950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff7b0005020 con 0x7ff7c8101310 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 -- start start 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81fa9b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faa0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faa3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a81f9facb0 con 0x55a81fa9b260 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a81f9a35b0 con 0x55a81faa0dc0 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a81f9f6c70 con 0x55a81fa9bc50 2024-04-25T00:36:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81fa9b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81fa9b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54088/0 (socket says 172.21.15.31:54088) 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.877+0000 7ff7c7fff640 1 -- 172.21.15.31:0/3285294873 learned_addr learned my addr 172.21.15.31:0/3285294873 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7beffd640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faa3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faa0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c7fff640 1 -- 172.21.15.31:0/3285294873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 msgr2=0x55a81faa0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faa0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c7fff640 1 -- 172.21.15.31:0/3285294873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 msgr2=0x55a81faa3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faa3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c7fff640 1 -- 172.21.15.31:0/3285294873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a81f70f210 con 0x55a81fa9b260 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faa0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7beffd640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faa3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c57fa640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff7b40a2c30 0x7ff7b40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81fa9b680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009cd0 tx=0x7ff7c0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838597dee1571444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff7b40a2c30 0x7ff7b40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7c57fa640 1 -- 172.21.15.31:0/350444950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff7b00cf3d0 con 0x7ff7c8101310 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7bdffb640 1 -- 172.21.15.31:0/3285294873 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0048070 con 0x55a81fa9b260 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7bdffb640 1 -- 172.21.15.31:0/3285294873 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c0036070 con 0x55a81fa9b260 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.878+0000 7ff7bdffb640 1 -- 172.21.15.31:0/3285294873 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0004600 con 0x55a81fa9b260 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/3285294873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 msgr2=0x55a81fa9b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff7b40a2c30 0x7ff7b40a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a81fa9c050 tx=0x7ff7b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81fa9b680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009cd0 tx=0x7ff7c0009a20 comp rx=0 tx=0).stop 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/3285294873 shutdown_connections 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faa3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faa0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/3285294873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81fa9b680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/3285294873 >> 172.21.15.31:0/3285294873 conn(0x55a81fa924c0 msgr2=0x55a81fa928c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/3285294873 shutdown_connections 2024-04-25T00:36:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/3285294873 wait complete. 2024-04-25T00:36:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.879+0000 7ff7d0b6e740 1 -- start start 2024-04-25T00:36:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 Processor -- start 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9b260 0x55a81faa56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 0x55a81faa5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81faa0dc0 0x55a81faacc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a81faa3d10 con 0x55a81fa9bc50 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a81f9b6b50 con 0x55a81fa9b260 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a81faa3e90 con 0x55a81faa0dc0 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7d0b6e740 10 client.0 _ll_register_callbacks cb 0x55a81f9f1ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 0x55a81faa5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 0x55a81faa5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54096/0 (socket says 172.21.15.31:54096) 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 -- 172.21.15.31:0/1566843825 learned_addr learned my addr 172.21.15.31:0/1566843825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9b260 0x55a81faa56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81faa0dc0 0x55a81faacc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 -- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81faa0dc0 msgr2=0x55a81faacc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81faa0dc0 0x55a81faacc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 -- 172.21.15.31:0/1566843825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9b260 msgr2=0x55a81faa56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9b260 0x55a81faa56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 -- 172.21.15.31:0/1566843825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a81f7db620 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7beffd640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 0x55a81faa5bf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0098060 tx=0x7ff7b009ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52808cdd213f2a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.880+0000 7ff7a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/1566843825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0005680 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/1566843825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff7b00a43e0 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c0036070 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/1566843825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7b009c300 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 10 client.15855 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a81faaffd0 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/1566843825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c0036070 con 0x55a81fa9bc50 2024-04-25T00:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/1566843825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff7b009b050 con 0x55a81fa9bc50 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 20 client.15855 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fab1670 0x55a81fab3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 4 client.15855 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.881+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a81fa081f0 con 0x55a81fab1670 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.882+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fab1670 0x55a81fab3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.882+0000 7ff7a6ffd640 10 client.15855 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.882+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fab1670 0x55a81fab3ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aab0e067da12773 server_cookie=eec7e4da4fe7bdba in_seq=0 out_seq=0 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/1566843825 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a81fa081f0 con 0x55a81fab1670 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7a6ffd640 10 client.15855 handle_command_reply: tid=0 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 client.15855 shutdown 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 msgr2=0x55a81faa5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 0x55a81faa5bf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0098060 tx=0x7ff7b009ba90 comp rx=0 tx=0).stop 2024-04-25T00:36:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 shutdown_connections 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fab1670 0x55a81fab3ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81faa0dc0 0x55a81faacc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9bc50 0x55a81faa5bf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1566843825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9b260 0x55a81faa56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 >> 172.21.15.31:0/1566843825 conn(0x55a81fa924c0 msgr2=0x55a81fa915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7a6ffd640 0 client.15855 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 shutdown_connections 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1566843825 wait complete. 2024-04-25T00:36:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.883+0000 7ff7d0b6e740 20 client.15855 trim_cache size 0 max 16384 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 Processor -- start 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 -- start start 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9b260 0x55a81fa9d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9bc50 0x55a81fa9d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 0x55a81fa95470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81a67a0 con 0x55a81faa0dc0 2024-04-25T00:36:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81a82e0 con 0x55a81fa9bc50 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81a6e70 con 0x55a81fa9b260 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 0x55a81fa95470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 0x55a81fa95470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54100/0 (socket says 172.21.15.31:54100) 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 -- 172.21.15.31:0/1518727525 learned_addr learned my addr 172.21.15.31:0/1518727525 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 -- 172.21.15.31:0/1518727525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9b260 msgr2=0x55a81fa9d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9b260 0x55a81fa9d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9b260 0x55a81fa9d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 -- 172.21.15.31:0/1518727525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9bc50 msgr2=0x55a81fa9d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9bc50 0x55a81fa9d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 -- 172.21.15.31:0/1518727525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a81fa93230 con 0x55a81faa0dc0 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9b260 0x55a81fa9d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 0x55a81fa95470 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009430 tx=0x7ff7c0031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4021377c890007de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.899+0000 7ff7bd7fa640 1 -- 172.21.15.31:0/1518727525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0051070 con 0x55a81faa0dc0 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7bd7fa640 1 -- 172.21.15.31:0/1518727525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c0036070 con 0x55a81faa0dc0 2024-04-25T00:36:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7bd7fa640 1 -- 172.21.15.31:0/1518727525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0048040 con 0x55a81faa0dc0 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1518727525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 msgr2=0x55a81fa95470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 0x55a81fa95470 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009430 tx=0x7ff7c0031670 comp rx=0 tx=0).stop 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1518727525 shutdown_connections 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81faa0dc0 0x55a81fa95470 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81fa9bc50 0x55a81fa9d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/1518727525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9b260 0x55a81fa9d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1518727525 >> 172.21.15.31:0/1518727525 conn(0x55a81fa924c0 msgr2=0x55a81fa928a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1518727525 shutdown_connections 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/1518727525 wait complete. 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.900+0000 7ff7d0b6e740 1 -- start start 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 Processor -- start 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81faa9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faaa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faaaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81a7c10 con 0x55a81fa9b260 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81a7540 con 0x55a81faa0dc0 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81a4780 con 0x55a81fa9bc50 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7d0b6e740 10 client.0 _ll_register_callbacks cb 0x7ff7b000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81faa9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81faa9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54116/0 (socket says 172.21.15.31:54116) 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 -- 172.21.15.31:0/2743989013 learned_addr learned my addr 172.21.15.31:0/2743989013 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7beffd640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faaa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faaaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 -- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 msgr2=0x55a81faaa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faaa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 -- 172.21.15.31:0/2743989013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 msgr2=0x55a81faaaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faaaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c77fe640 1 -- 172.21.15.31:0/2743989013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a81faab5e0 con 0x55a81fa9b260 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.901+0000 7ff7c7fff640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faaaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7c77fe640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81faa9b40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55a81fa9c6c0 tx=0x7ff7b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa89dc29d7be48f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/2743989013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b801fa70 con 0x55a81fa9b260 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/2743989013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff7b801fc10 con 0x55a81fa9b260 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c0036070 con 0x55a81fa9b260 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/2743989013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7b8037050 con 0x55a81fa9b260 2024-04-25T00:36:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 10 client.15861 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a81fb17ec0 con 0x55a81fa9b260 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/2743989013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c0036070 con 0x55a81fa9b260 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/2743989013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff7b8030520 con 0x55a81fa9b260 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 10 client.15861 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 20 client.15861 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fb188f0 0x55a81fb1ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 4 client.15861 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.902+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a81fa02540 con 0x55a81fb188f0 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.903+0000 7ff7beffd640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fb188f0 0x55a81fb1ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.903+0000 7ff7beffd640 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fb188f0 0x55a81fb1ad10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efae7e0c12f45b75 server_cookie=5c411c37fc47494 in_seq=0 out_seq=0 2024-04-25T00:36:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.903+0000 7ff7a6ffd640 10 client.15861 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7a6ffd640 1 -- 172.21.15.31:0/2743989013 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a81fa02540 con 0x55a81fb188f0 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7a6ffd640 10 client.15861 handle_command_reply: tid=0 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 client.15861 shutdown 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 msgr2=0x55a81faa9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81faa9b40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55a81fa9c6c0 tx=0x7ff7b80046c0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 shutdown_connections 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a81fb188f0 0x55a81fb1ad10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a81faa0dc0 0x55a81faaaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a81fa9bc50 0x55a81faaa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7a6ffd640 0 client.15861 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 --2- 172.21.15.31:0/2743989013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a81fa9b260 0x55a81faa9b40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 >> 172.21.15.31:0/2743989013 conn(0x55a81fa924c0 msgr2=0x55a81faa5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 shutdown_connections 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 1 -- 172.21.15.31:0/2743989013 wait complete. 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.919+0000 7ff7d0b6e740 20 client.15861 trim_cache size 0 max 16384 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:36:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:36:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff7b40a2c30 msgr2=0x7ff7b40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff7b40a2c30 0x7ff7b40a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a81fa9c050 tx=0x7ff7b8005f90 comp rx=0 tx=0).stop 2024-04-25T00:36:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 msgr2=0x7ff7c8197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8197db0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000e9f0 tx=0x7ff7b000edf0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 shutdown_connections 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c8101cb0 0x7ff7c819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff7b40a2c30 0x7ff7b40a50f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c8101310 0x7ff7c8197db0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 --2- 172.21.15.31:0/350444950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8100950 0x7ff7c8197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 >> 172.21.15.31:0/350444950 conn(0x7ff7c80fb3c0 msgr2=0x7ff7c80fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 shutdown_connections 2024-04-25T00:36:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:24.920+0000 7ff7cdd7e640 1 -- 172.21.15.31:0/350444950 wait complete. 2024-04-25T00:36:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:24 smithi031 ceph-mon[29167]: pgmap v536: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:25.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:24 smithi169 ceph-mon[33112]: pgmap v536: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:25.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:24 smithi040 ceph-mon[32945]: pgmap v536: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:36:27.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:26 smithi031 ceph-mon[29167]: pgmap v537: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:36:27.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:26 smithi169 ceph-mon[33112]: pgmap v537: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:36:27.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:26 smithi040 ceph-mon[32945]: pgmap v537: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:36:29.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:28 smithi040 ceph-mon[32945]: pgmap v538: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:36:29.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:28 smithi031 ceph-mon[29167]: pgmap v538: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:36:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:28 smithi169 ceph-mon[33112]: pgmap v538: 97 pgs: 97 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:36:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:30 smithi031 ceph-mon[29167]: pgmap v539: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:31.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:30 smithi169 ceph-mon[33112]: pgmap v539: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:30 smithi040 ceph-mon[32945]: pgmap v539: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:33.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:32 smithi031 ceph-mon[29167]: pgmap v540: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:33.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:32 smithi169 ceph-mon[33112]: pgmap v540: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:32 smithi040 ceph-mon[32945]: pgmap v540: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:35.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:34 smithi031 ceph-mon[29167]: pgmap v541: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:35.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:34 smithi169 ceph-mon[33112]: pgmap v541: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:34 smithi040 ceph-mon[32945]: pgmap v541: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:36 smithi031 ceph-mon[29167]: pgmap v542: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:36 smithi169 ceph-mon[33112]: pgmap v542: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:36 smithi040 ceph-mon[32945]: pgmap v542: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 -- 172.21.15.31:0/4181840800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de01006a0 msgr2=0x7f2de0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 --2- 172.21.15.31:0/4181840800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de01006a0 0x7f2de0100ac0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009900 tx=0x7f2dd002a410 comp rx=0 tx=0).stop 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 -- 172.21.15.31:0/4181840800 shutdown_connections 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 --2- 172.21.15.31:0/4181840800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2de0102250 0x7f2de0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 --2- 172.21.15.31:0/4181840800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2de0101890 0x7f2de0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 --2- 172.21.15.31:0/4181840800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de01006a0 0x7f2de0100ac0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 -- 172.21.15.31:0/4181840800 >> 172.21.15.31:0/4181840800 conn(0x7f2de00fbc10 msgr2=0x7f2de00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 -- 172.21.15.31:0/4181840800 shutdown_connections 2024-04-25T00:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.020+0000 7f2de8493640 1 -- 172.21.15.31:0/4181840800 wait complete. 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 Processor -- start 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 -- start start 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2de01006a0 0x7f2de0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 0x7f2de0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2de0102250 0x7f2de019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2de010c1d0 con 0x7f2de0101890 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2de010c050 con 0x7f2de01006a0 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de8493640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2de010c350 con 0x7f2de0102250 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de5a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 0x7f2de0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de5a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 0x7f2de0197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39854/0 (socket says 172.21.15.31:39854) 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de5a2e640 1 -- 172.21.15.31:0/1694836763 learned_addr learned my addr 172.21.15.31:0/1694836763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de6a30640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2de0102250 0x7f2de019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de622f640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2de01006a0 0x7f2de0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de5a2e640 1 -- 172.21.15.31:0/1694836763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2de0102250 msgr2=0x7f2de019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.021+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2de0102250 0x7f2de019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2de5a2e640 1 -- 172.21.15.31:0/1694836763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2de01006a0 msgr2=0x7f2de0197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2de01006a0 0x7f2de0197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2de5a2e640 1 -- 172.21.15.31:0/1694836763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de019f550 con 0x7f2de0101890 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 0x7f2de0197e70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400d950 tx=0x7f2dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2ddf7fe640 1 -- 172.21.15.31:0/1694836763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4017070 con 0x7f2de0101890 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de019f840 con 0x7f2de0101890 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2ddf7fe640 1 -- 172.21.15.31:0/1694836763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd4009c30 con 0x7f2de0101890 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2ddf7fe640 1 -- 172.21.15.31:0/1694836763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd400f620 con 0x7f2de0101890 2024-04-25T00:36:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.022+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de019fd90 con 0x7f2de0101890 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.024+0000 7f2ddf7fe640 1 -- 172.21.15.31:0/1694836763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2dd40109a0 con 0x7f2de0101890 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.025+0000 7f2deb19d740 1 Processor -- start 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.025+0000 7f2deb19d740 1 -- start start 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.025+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e08e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.025+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08e840 0x55dc5e093340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.025+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e095e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2deb19d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dc5dff1cb0 con 0x55dc5e0939b0 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2deb19d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dc5df9a5b0 con 0x55dc5e08de50 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2deb19d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dc5dfedc70 con 0x55dc5e08e840 2024-04-25T00:36:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2ddf7fe640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2dc80a2c30 0x7f2dc80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e095e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de6a30640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2dc80a2c30 0x7f2dc80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e095e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39868/0 (socket says 172.21.15.31:39868) 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 -- 172.21.15.31:0/4119620444 learned_addr learned my addr 172.21.15.31:0/4119620444 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de622f640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08e840 0x55dc5e093340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de6a30640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e08e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2ddf7fe640 1 -- 172.21.15.31:0/1694836763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2dd40ced20 con 0x7f2de0101890 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 -- 172.21.15.31:0/4119620444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08e840 msgr2=0x55dc5e093340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08e840 0x55dc5e093340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 -- 172.21.15.31:0/4119620444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 msgr2=0x55dc5e08e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e08e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 -- 172.21.15.31:0/4119620444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5dd06210 con 0x55dc5e0939b0 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de622f640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08e840 0x55dc5e093340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de6a30640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e08e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de6a30640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2dc80a2c30 0x7f2dc80a50f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009fd0 tx=0x7f2dd8009ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.026+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e095e40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40cefc0 tx=0x7f2dd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0129e69ec74ca70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2ddcbfe640 1 -- 172.21.15.31:0/4119620444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4020040 con 0x55dc5e0939b0 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2ddcbfe640 1 -- 172.21.15.31:0/4119620444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd409b4f0 con 0x55dc5e0939b0 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2ddcbfe640 1 -- 172.21.15.31:0/4119620444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4010680 con 0x55dc5e0939b0 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 -- 172.21.15.31:0/4119620444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 msgr2=0x55dc5e095e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e095e40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40cefc0 tx=0x7f2dd4005880 comp rx=0 tx=0).stop 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 -- 172.21.15.31:0/4119620444 shutdown_connections 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e095e40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08e840 0x55dc5e093340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 --2- 172.21.15.31:0/4119620444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e08e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.027+0000 7f2deb19d740 1 -- 172.21.15.31:0/4119620444 >> 172.21.15.31:0/4119620444 conn(0x55dc5e0894c0 msgr2=0x55dc5e0898c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 -- 172.21.15.31:0/4119620444 shutdown_connections 2024-04-25T00:36:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 -- 172.21.15.31:0/4119620444 wait complete. 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 -- start start 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 Processor -- start 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dc5e096900 con 0x55dc5e0939b0 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dc5dfadb50 con 0x55dc5e08e840 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dc5e096a80 con 0x55dc5e08de50 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2deb19d740 10 client.0 _ll_register_callbacks cb 0x55dc5dfe8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2de6a30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2de6a30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39872/0 (socket says 172.21.15.31:39872) 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2de6a30640 1 -- 172.21.15.31:0/325313604 learned_addr learned my addr 172.21.15.31:0/325313604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.028+0000 7f2de622f640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2de6a30640 1 -- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08de50 msgr2=0x55dc5e098250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2de6a30640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2de6a30640 1 -- 172.21.15.31:0/325313604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 msgr2=0x55dc5e098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2de6a30640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2de6a30640 1 -- 172.21.15.31:0/325313604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5ddd2620 con 0x55dc5e0939b0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2de6a30640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f7d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8010bc0 tx=0x7f2dd8009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2d17c504360dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2dc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/325313604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8025070 con 0x55dc5e0939b0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/325313604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2dd8004b20 con 0x55dc5e0939b0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd409b4f0 con 0x55dc5e0939b0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/325313604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2dd8004d20 con 0x55dc5e0939b0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2deb19d740 10 client.15873 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.029+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc5e0a2c00 con 0x55dc5e0939b0 2024-04-25T00:36:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/325313604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd409b4f0 con 0x55dc5e0939b0 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/325313604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2dd800f040 con 0x55dc5e0939b0 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2deb19d740 10 client.15873 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2deb19d740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2deb19d740 20 client.15873 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2deb19d740 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e0a4310 0x55dc5e0a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2deb19d740 4 client.15873 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc5dfff1f0 con 0x55dc5e0a4310 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.030+0000 7f2de622f640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e0a4310 0x55dc5e0a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.031+0000 7f2dc5ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.031+0000 7f2de622f640 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e0a4310 0x55dc5e0a6740 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307812e3f460d6ec server_cookie=8fda884668e9d6fb in_seq=0 out_seq=0 2024-04-25T00:36:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.031+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/325313604 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc5dfff1f0 con 0x55dc5e0a4310 2024-04-25T00:36:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.031+0000 7f2dc5ffb640 10 client.15873 handle_command_reply: tid=0 2024-04-25T00:36:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.031+0000 7f2deb19d740 1 client.15873 shutdown 2024-04-25T00:36:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 msgr2=0x55dc5e09f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f7d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8010bc0 tx=0x7f2dd8009600 comp rx=0 tx=0).stop 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 shutdown_connections 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e0a4310 0x55dc5e0a6740 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f7d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 --2- 172.21.15.31:0/325313604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2dc5ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 >> 172.21.15.31:0/325313604 conn(0x55dc5e0894c0 msgr2=0x55dc5e0885c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 shutdown_connections 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 1 -- 172.21.15.31:0/325313604 wait complete. 2024-04-25T00:36:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.032+0000 7f2deb19d740 20 client.15873 trim_cache size 0 max 16384 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 Processor -- start 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 -- start start 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01a8c40 con 0x55dc5e08e840 2024-04-25T00:36:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01aa780 con 0x55dc5e08de50 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2deb19d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01a9310 con 0x55dc5e0939b0 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2de5a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2de5a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39876/0 (socket says 172.21.15.31:39876) 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2de5a2e640 1 -- 172.21.15.31:0/2166369119 learned_addr learned my addr 172.21.15.31:0/2166369119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.047+0000 7f2de622f640 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de6a30640 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de5a2e640 1 -- 172.21.15.31:0/2166369119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 msgr2=0x55dc5e09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de5a2e640 1 -- 172.21.15.31:0/2166369119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 msgr2=0x55dc5e098250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de5a2e640 1 -- 172.21.15.31:0/2166369119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5e092f60 con 0x55dc5e08e840 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de622f640 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4097ec0 tx=0x7f2dd4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81997106fb3663b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2dc7fff640 1 -- 172.21.15.31:0/2166369119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40b4070 con 0x55dc5e08e840 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2dc7fff640 1 -- 172.21.15.31:0/2166369119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd409f8a0 con 0x55dc5e08e840 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.048+0000 7f2dc7fff640 1 -- 172.21.15.31:0/2166369119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4020040 con 0x55dc5e08e840 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- 172.21.15.31:0/2166369119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 msgr2=0x55dc5e098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4097ec0 tx=0x7f2dd4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- 172.21.15.31:0/2166369119 shutdown_connections 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08e840 0x55dc5e098790 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- 172.21.15.31:0/2166369119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08de50 0x55dc5e098250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- 172.21.15.31:0/2166369119 >> 172.21.15.31:0/2166369119 conn(0x55dc5e0894c0 msgr2=0x55dc5e0898a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- 172.21.15.31:0/2166369119 shutdown_connections 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- 172.21.15.31:0/2166369119 wait complete. 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- start start 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 Processor -- start 2024-04-25T00:36:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 0x55dc5e09cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01aa0b0 con 0x55dc5e08de50 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01a99e0 con 0x55dc5e08e840 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01a6c20 con 0x55dc5e0939b0 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2deb19d740 10 client.0 _ll_register_callbacks cb 0x7f2dd8010710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2de622f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 0x55dc5e09cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.049+0000 7f2de622f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 0x55dc5e09cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39882/0 (socket says 172.21.15.31:39882) 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 -- 172.21.15.31:0/1811846429 learned_addr learned my addr 172.21.15.31:0/1811846429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de6a30640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 -- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 msgr2=0x55dc5e09e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 -- 172.21.15.31:0/1811846429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 msgr2=0x55dc5e09d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e09d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 -- 172.21.15.31:0/1811846429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5e09e6c0 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de6a30640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2de622f640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 0x55dc5e09cb80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005ab0 tx=0x7f2dd0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5285353984bf2a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2dc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/1811846429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0049070 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/1811846429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2dd0045dc0 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd409f8a0 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/1811846429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2dd0055050 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2deb19d740 10 client.15879 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.050+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc5e09fa70 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/1811846429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd409f8a0 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/1811846429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2dd0031700 con 0x55dc5e08de50 2024-04-25T00:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2deb19d740 10 client.15879 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2deb19d740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2deb19d740 20 client.15879 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2deb19d740 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e10b040 0x55dc5e10d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2deb19d740 4 client.15879 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc5dff9540 con 0x55dc5e10b040 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e10b040 0x55dc5e10d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.052+0000 7f2dc5ffb640 10 client.15879 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.051+0000 7f2de5a2e640 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e10b040 0x55dc5e10d460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32764a3a9e2e1b78 server_cookie=5d67919fff765ea8 in_seq=0 out_seq=0 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.068+0000 7f2dc5ffb640 1 -- 172.21.15.31:0/1811846429 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc5dff9540 con 0x55dc5e10b040 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.068+0000 7f2dc5ffb640 10 client.15879 handle_command_reply: tid=0 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.068+0000 7f2deb19d740 1 client.15879 shutdown 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 msgr2=0x55dc5e09cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 0x55dc5e09cb80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005ab0 tx=0x7f2dd0004290 comp rx=0 tx=0).stop 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 shutdown_connections 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55dc5e10b040 0x55dc5e10d460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc5e0939b0 0x55dc5e09e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2dc5ffb640 0 client.15879 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc5e08e840 0x55dc5e09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 --2- 172.21.15.31:0/1811846429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc5e08de50 0x55dc5e09cb80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 >> 172.21.15.31:0/1811846429 conn(0x55dc5e0894c0 msgr2=0x55dc5e08fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 shutdown_connections 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 1 -- 172.21.15.31:0/1811846429 wait complete. 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2deb19d740 20 client.15879 trim_cache size 0 max 16384 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:36:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:36:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2dc80a2c30 msgr2=0x7f2dc80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2dc80a2c30 0x7f2dc80a50f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009fd0 tx=0x7f2dd8009ba0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 msgr2=0x7f2de0197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 0x7f2de0197e70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400d950 tx=0x7f2dd400de10 comp rx=0 tx=0).stop 2024-04-25T00:36:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 shutdown_connections 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2de0102250 0x7f2de019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2dc80a2c30 0x7f2dc80a50f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2de0101890 0x7f2de0197e70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 --2- 172.21.15.31:0/1694836763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2de01006a0 0x7f2de0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.069+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 >> 172.21.15.31:0/1694836763 conn(0x7f2de00fbc10 msgr2=0x7f2de00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.070+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 shutdown_connections 2024-04-25T00:36:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:38.070+0000 7f2de8493640 1 -- 172.21.15.31:0/1694836763 wait complete. 2024-04-25T00:36:39.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:38 smithi031 ceph-mon[29167]: pgmap v543: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:36:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:38 smithi169 ceph-mon[33112]: pgmap v543: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:36:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:38 smithi040 ceph-mon[32945]: pgmap v543: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:36:41.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:40 smithi031 ceph-mon[29167]: pgmap v544: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:41.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:40 smithi169 ceph-mon[33112]: pgmap v544: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:40 smithi040 ceph-mon[32945]: pgmap v544: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:36:43.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:42 smithi031 ceph-mon[29167]: pgmap v545: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-04-25T00:36:43.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:42 smithi169 ceph-mon[33112]: pgmap v545: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-04-25T00:36:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:42 smithi040 ceph-mon[32945]: pgmap v545: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-04-25T00:36:45.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:44 smithi031 ceph-mon[29167]: pgmap v546: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:45.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:44 smithi169 ceph-mon[33112]: pgmap v546: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:44 smithi040 ceph-mon[32945]: pgmap v546: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:46 smithi031 ceph-mon[29167]: pgmap v547: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:36:47.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:46 smithi169 ceph-mon[33112]: pgmap v547: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:36:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:46 smithi040 ceph-mon[32945]: pgmap v547: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:36:49.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:48 smithi031 ceph-mon[29167]: pgmap v548: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:49.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:48 smithi169 ceph-mon[33112]: pgmap v548: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:48 smithi040 ceph-mon[32945]: pgmap v548: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 -- 172.21.15.31:0/2973604105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 msgr2=0x7f05c4100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 --2- 172.21.15.31:0/2973604105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4100ae0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009900 tx=0x7f05b402a410 comp rx=0 tx=0).stop 2024-04-25T00:36:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 -- 172.21.15.31:0/2973604105 shutdown_connections 2024-04-25T00:36:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 --2- 172.21.15.31:0/2973604105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05c4102270 0x7f05c4108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 --2- 172.21.15.31:0/2973604105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c41018b0 0x7f05c4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 --2- 172.21.15.31:0/2973604105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4100ae0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 -- 172.21.15.31:0/2973604105 >> 172.21.15.31:0/2973604105 conn(0x7f05c40fbc30 msgr2=0x7f05c40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 -- 172.21.15.31:0/2973604105 shutdown_connections 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.165+0000 7f05c9334640 1 -- 172.21.15.31:0/2973604105 wait complete. 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 Processor -- start 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 -- start start 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05c41018b0 0x7f05c419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c4102270 0x7f05c41a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05c410c1f0 con 0x7f05c41006c0 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05c410c070 con 0x7f05c4102270 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c9334640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05c410c370 con 0x7f05c41018b0 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56462/0 (socket says 172.21.15.31:56462) 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3279937399 learned_addr learned my addr 172.21.15.31:0/3279937399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05bbfff640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05c41018b0 0x7f05c419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c4102270 0x7f05c41a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3279937399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05c41018b0 msgr2=0x7f05c419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05c41018b0 0x7f05c419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3279937399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c4102270 msgr2=0x7f05c41a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c4102270 0x7f05c41a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.166+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3279937399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c41a1780 con 0x7f05c41006c0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.167+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4199b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f05b40314e0 tx=0x7f05b4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.167+0000 7f05c0ff9640 1 -- 172.21.15.31:0/3279937399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b403f030 con 0x7f05c41006c0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.167+0000 7f05c0ff9640 1 -- 172.21.15.31:0/3279937399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05b40044b0 con 0x7f05c41006c0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.167+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c41a1a10 con 0x7f05c41006c0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.167+0000 7f05c0ff9640 1 -- 172.21.15.31:0/3279937399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4035070 con 0x7f05c41006c0 2024-04-25T00:36:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.167+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c41a1f00 con 0x7f05c41006c0 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.169+0000 7f05c0ff9640 1 -- 172.21.15.31:0/3279937399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f05b4042030 con 0x7f05c41006c0 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.169+0000 7f05cc040740 1 Processor -- start 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 -- start start 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f35e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f2a9e91cb0 con 0x55f2a9f2de50 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f2a9e3a5b0 con 0x55f2a9f2e840 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05cc040740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2a9e8dc70 con 0x55f2a9f339b0 2024-04-25T00:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c0ff9640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f05b00a2c30 0x7f05b00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56476/0 (socket says 172.21.15.31:56476) 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c37fe640 1 -- 172.21.15.31:0/3530916457 learned_addr learned my addr 172.21.15.31:0/3530916457 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f05b00a2c30 0x7f05b00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05bbfff640 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f35e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.170+0000 7f05c0ff9640 1 -- 172.21.15.31:0/3279937399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f05b40ef540 con 0x7f05c41006c0 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 -- 172.21.15.31:0/3530916457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 msgr2=0x55f2a9f35e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f35e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 -- 172.21.15.31:0/3530916457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 msgr2=0x55f2a9f33340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f33340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 -- 172.21.15.31:0/3530916457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a9ba6210 con 0x55f2a9f2de50 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f33340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2e270 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f05bc012d80 tx=0x7f05bc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f72e3b181d3bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f05b00a2c30 0x7f05b00a50f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05bc010c40 tx=0x7f05bc00c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05b97fa640 1 -- 172.21.15.31:0/3530916457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc024070 con 0x55f2a9f2de50 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05b97fa640 1 -- 172.21.15.31:0/3530916457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05bc0125a0 con 0x55f2a9f2de50 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.171+0000 7f05b97fa640 1 -- 172.21.15.31:0/3530916457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc005280 con 0x55f2a9f2de50 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 -- 172.21.15.31:0/3530916457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 msgr2=0x55f2a9f2e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2e270 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f05bc012d80 tx=0x7f05bc010740 comp rx=0 tx=0).stop 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 -- 172.21.15.31:0/3530916457 shutdown_connections 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f35e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f33340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 --2- 172.21.15.31:0/3530916457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2e270 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 -- 172.21.15.31:0/3530916457 >> 172.21.15.31:0/3530916457 conn(0x55f2a9f294c0 msgr2=0x55f2a9f298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 -- 172.21.15.31:0/3530916457 shutdown_connections 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 -- 172.21.15.31:0/3530916457 wait complete. 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 -- start start 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.172+0000 7f05cc040740 1 Processor -- start 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2de50 0x55f2a9f38290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 0x55f2a9f387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f2a9f36900 con 0x55f2a9f2e840 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f2a9e4db50 con 0x55f2a9f2de50 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2a9f36a80 con 0x55f2a9f339b0 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05cc040740 10 client.0 _ll_register_callbacks cb 0x55f2a9e88ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 0x55f2a9f387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 0x55f2a9f387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56484/0 (socket says 172.21.15.31:56484) 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 -- 172.21.15.31:0/2389738915 learned_addr learned my addr 172.21.15.31:0/2389738915 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2de50 0x55f2a9f38290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05c37fe640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 -- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 msgr2=0x55f2a9f3f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 -- 172.21.15.31:0/2389738915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2de50 msgr2=0x55f2a9f38290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2de50 0x55f2a9f38290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 -- 172.21.15.31:0/2389738915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a9c72620 con 0x55f2a9f2e840 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05c37fe640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2de50 0x55f2a9f38290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05bbfff640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 0x55f2a9f387d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f05ac012700 tx=0x7f05ac012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71881ed8f923b26d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05aa7fc640 1 -- 172.21.15.31:0/2389738915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00f8c0 con 0x55f2a9f2e840 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.173+0000 7f05aa7fc640 1 -- 172.21.15.31:0/2389738915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f05ac00fa60 con 0x55f2a9f2e840 2024-04-25T00:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05bc0125a0 con 0x55f2a9f2e840 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05aa7fc640 1 -- 172.21.15.31:0/2389738915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05ac00fc60 con 0x55f2a9f2e840 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 10 client.15891 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2a9f42c50 con 0x55f2a9f2e840 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05aa7fc640 1 -- 172.21.15.31:0/2389738915 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05bc0125a0 con 0x55f2a9f2e840 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05aa7fc640 1 -- 172.21.15.31:0/2389738915 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05ac010430 con 0x55f2a9f2e840 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 10 client.15891 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 20 client.15891 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9f44390 0x55f2a9f467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 4 client.15891 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2a9e9f1f0 con 0x55f2a9f44390 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.174+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9f44390 0x55f2a9f467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.175+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9f44390 0x55f2a9f467f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f22a5483a9a4f3c server_cookie=6404ffbc785110d4 in_seq=0 out_seq=0 2024-04-25T00:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.175+0000 7f05aa7fc640 10 client.15891 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05aa7fc640 1 -- 172.21.15.31:0/2389738915 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2a9e9f1f0 con 0x55f2a9f44390 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05aa7fc640 10 client.15891 handle_command_reply: tid=0 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 client.15891 shutdown 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 msgr2=0x55f2a9f387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 0x55f2a9f387d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f05ac012700 tx=0x7f05ac012bc0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 shutdown_connections 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9f44390 0x55f2a9f467f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2e840 0x55f2a9f387d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05aa7fc640 0 client.15891 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 --2- 172.21.15.31:0/2389738915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2de50 0x55f2a9f38290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 >> 172.21.15.31:0/2389738915 conn(0x55f2a9f294c0 msgr2=0x55f2a9f285c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 shutdown_connections 2024-04-25T00:36:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 1 -- 172.21.15.31:0/2389738915 wait complete. 2024-04-25T00:36:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.176+0000 7f05cc040740 20 client.15891 trim_cache size 0 max 16384 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 Processor -- start 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 -- start start 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2e840 0x55f2a9f30250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41aae20 con 0x55f2a9f2de50 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41ac960 con 0x55f2a9f339b0 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.191+0000 7f05cc040740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41ab4f0 con 0x55f2a9f2e840 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56494/0 (socket says 172.21.15.31:56494) 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3821844803 learned_addr learned my addr 172.21.15.31:0/3821844803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05bbfff640 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2e840 0x55f2a9f30250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3821844803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2e840 msgr2=0x55f2a9f30250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2e840 0x55f2a9f30250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3821844803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f339b0 msgr2=0x55f2a9f3f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 -- 172.21.15.31:0/3821844803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a9f32fc0 con 0x55f2a9f2de50 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2fd10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f05b40bbdb0 tx=0x7f05b4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d33b4de8680d57b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05b8ff9640 1 -- 172.21.15.31:0/3821844803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4049040 con 0x55f2a9f2de50 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05b8ff9640 1 -- 172.21.15.31:0/3821844803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05b40c0120 con 0x55f2a9f2de50 2024-04-25T00:36:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.192+0000 7f05b8ff9640 1 -- 172.21.15.31:0/3821844803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40bc080 con 0x55f2a9f2de50 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 -- 172.21.15.31:0/3821844803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 msgr2=0x55f2a9f2fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2fd10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f05b40bbdb0 tx=0x7f05b4005e60 comp rx=0 tx=0).stop 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 -- 172.21.15.31:0/3821844803 shutdown_connections 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2e840 0x55f2a9f30250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 --2- 172.21.15.31:0/3821844803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f2de50 0x55f2a9f2fd10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 -- 172.21.15.31:0/3821844803 >> 172.21.15.31:0/3821844803 conn(0x55f2a9f294c0 msgr2=0x55f2a9f298a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 -- 172.21.15.31:0/3821844803 shutdown_connections 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.193+0000 7f05cc040740 1 -- 172.21.15.31:0/3821844803 wait complete. 2024-04-25T00:36:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 -- start start 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 Processor -- start 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2de50 0x55f2a9f3cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41ac290 con 0x55f2a9f339b0 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41abbc0 con 0x55f2a9f2e840 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41a8e00 con 0x55f2a9f2de50 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05cc040740 10 client.0 _ll_register_callbacks cb 0x7f05ac009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56500/0 (socket says 172.21.15.31:56500) 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2de50 0x55f2a9f3cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 -- 172.21.15.31:0/3137454362 learned_addr learned my addr 172.21.15.31:0/3137454362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05bbfff640 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 -- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2de50 msgr2=0x55f2a9f3cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2de50 0x55f2a9f3cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 -- 172.21.15.31:0/3137454362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 msgr2=0x55f2a9f3d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f3d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.194+0000 7f05c37fe640 1 -- 172.21.15.31:0/3137454362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a9f3e6e0 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05c37fe640 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3e0f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0096e0 tx=0x7f05bc005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a74b886602a872b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05aa7fc640 1 -- 172.21.15.31:0/3137454362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc024050 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05aa7fc640 1 -- 172.21.15.31:0/3137454362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f05bc005000 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b40c0120 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05aa7fc640 1 -- 172.21.15.31:0/3137454362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05bc005200 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 10 client.15897 fetch_fsmap learned FSMap version 26 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2a9f3fa60 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05aa7fc640 1 -- 172.21.15.31:0/3137454362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05b40c0120 con 0x55f2a9f339b0 2024-04-25T00:36:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05aa7fc640 1 -- 172.21.15.31:0/3137454362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05bc00f040 con 0x55f2a9f339b0 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 10 client.15897 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 20 client.15897 populate_metadata read hostname 'smithi031' 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9faada0 0x55f2a9fad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 4 client.15897 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.195+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2a9e99540 con 0x55f2a9faada0 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.196+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9faada0 0x55f2a9fad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.196+0000 7f05c2ffd640 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9faada0 0x55f2a9fad190 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4670cd44173783 server_cookie=612d62e9d0bdbfeb in_seq=0 out_seq=0 2024-04-25T00:36:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.196+0000 7f05aa7fc640 10 client.15897 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.209+0000 7f05aa7fc640 1 -- 172.21.15.31:0/3137454362 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2a9e99540 con 0x55f2a9faada0 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05aa7fc640 10 client.15897 handle_command_reply: tid=0 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 client.15897 shutdown 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 msgr2=0x55f2a9f3e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3e0f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0096e0 tx=0x7f05bc005c40 comp rx=0 tx=0).stop 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 shutdown_connections 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f2a9faada0 0x55f2a9fad190 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2a9f339b0 0x55f2a9f3e0f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2a9f2e840 0x55f2a9f3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 --2- 172.21.15.31:0/3137454362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a9f2de50 0x55f2a9f3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05aa7fc640 0 client.15897 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 >> 172.21.15.31:0/3137454362 conn(0x55f2a9f294c0 msgr2=0x55f2a9f38160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 shutdown_connections 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 1 -- 172.21.15.31:0/3137454362 wait complete. 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.210+0000 7f05cc040740 20 client.15897 trim_cache size 0 max 16384 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:36:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:36:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f05b00a2c30 msgr2=0x7f05b00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f05b00a2c30 0x7f05b00a50f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05bc010c40 tx=0x7f05bc00c000 comp rx=0 tx=0).stop 2024-04-25T00:36:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 msgr2=0x7f05c4199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4199b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f05b40314e0 tx=0x7f05b4031a30 comp rx=0 tx=0).stop 2024-04-25T00:36:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 shutdown_connections 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c4102270 0x7f05c41a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05c41018b0 0x7f05c419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f05b00a2c30 0x7f05b00a50f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 --2- 172.21.15.31:0/3279937399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05c41006c0 0x7f05c4199b60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 >> 172.21.15.31:0/3279937399 conn(0x7f05c40fbc30 msgr2=0x7f05c40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 shutdown_connections 2024-04-25T00:36:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:36:51.211+0000 7f05c9334640 1 -- 172.21.15.31:0/3279937399 wait complete. 2024-04-25T00:36:51.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:50 smithi031 ceph-mon[29167]: pgmap v549: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:51.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:50 smithi169 ceph-mon[33112]: pgmap v549: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:51.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:50 smithi040 ceph-mon[32945]: pgmap v549: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:36:52.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:36:52.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:36:52.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:36:52.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:51 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:36:52.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:51 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:36:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:51 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:36:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:52 smithi031 ceph-mon[29167]: pgmap v550: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:53.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:52 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T00:36:53.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:52 smithi031 ceph-mon[29167]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T00:36:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:52 smithi169 ceph-mon[33112]: pgmap v550: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:52 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T00:36:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:52 smithi169 ceph-mon[33112]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T00:36:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:52 smithi040 ceph-mon[32945]: pgmap v550: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:36:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:52 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T00:36:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:52 smithi040 ceph-mon[32945]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T00:36:55.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:54 smithi031 ceph-mon[29167]: pgmap v551: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:36:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:54 smithi169 ceph-mon[33112]: pgmap v551: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:36:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:54 smithi040 ceph-mon[32945]: pgmap v551: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-04-25T00:36:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:56 smithi031 ceph-mon[29167]: pgmap v552: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:56 smithi169 ceph-mon[33112]: pgmap v552: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:56 smithi040 ceph-mon[32945]: pgmap v552: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:36:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:36:58 smithi031 ceph-mon[29167]: pgmap v553: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:59.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:36:58 smithi169 ceph-mon[33112]: pgmap v553: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:36:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:36:58 smithi040 ceph-mon[32945]: pgmap v553: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:37:01.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:01 smithi031 ceph-mon[29167]: pgmap v554: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:01.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:01 smithi169 ceph-mon[33112]: pgmap v554: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:01 smithi040 ceph-mon[32945]: pgmap v554: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:02.888 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.888 INFO:tasks.workunit.client.0.smithi031.stdout:251/600: open fd = 3 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:252/600: open fd = 3 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:253/600: open fd = 3 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:254/600: open fd = 3 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.889 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:255/600: open fd = 3 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:256/600: open fd = 3 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:257/600: open fd = 3 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.890 INFO:tasks.workunit.client.0.smithi031.stdout:258/600: open fd = 3 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:259/600: open fd = 3 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:260/600: open fd = 3 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:261/600: open fd = 3 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.891 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:262/600: open fd = 3 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:263/600: open fd = 3 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:264/600: open fd = 3 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.892 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:265/600: open fd = 3 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:266/600: open fd = 3 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:267/600: open fd = 3 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:268/600: open fd = 3 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.893 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:269/600: open fd = 3 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:270/600: open fd = 3 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:271/600: open fd = 3 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.894 INFO:tasks.workunit.client.0.smithi031.stdout:272/600: open fd = 3 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:273/600: open fd = 3 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:274/600: open fd = 3 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:275/600: open fd = 3 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.895 INFO:tasks.workunit.client.0.smithi031.stdout:276/600: open fd = 3 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:277/600: open fd = 3 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:278/600: open fd = 3 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:279/600: open fd = 3 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.896 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:280/600: open fd = 3 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:281/600: open fd = 3 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:282/600: open fd = 3 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:283/600: open fd = 3 2024-04-25T00:37:02.897 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:284/600: open fd = 3 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:285/600: open fd = 3 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:286/600: open fd = 3 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.898 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:287/600: open fd = 3 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:288/600: open fd = 3 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:289/600: open fd = 3 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:290/600: open fd = 3 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.899 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:291/600: open fd = 3 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:292/600: open fd = 3 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:293/600: open fd = 3 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:294/600: open fd = 3 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.900 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:295/600: open fd = 3 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:296/600: open fd = 3 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:297/600: open fd = 3 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:298/600: open fd = 3 2024-04-25T00:37:02.901 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:299/600: open fd = 3 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:300/600: open fd = 3 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:301/600: open fd = 3 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.902 INFO:tasks.workunit.client.0.smithi031.stdout:302/600: open fd = 3 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:303/600: open fd = 3 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:304/600: open fd = 3 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:305/600: open fd = 3 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.903 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:306/600: open fd = 3 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:307/600: open fd = 3 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:308/600: open fd = 3 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:309/600: open fd = 3 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.904 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:310/600: open fd = 3 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:311/600: open fd = 3 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:312/600: open fd = 3 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:02.905 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:37:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:03 smithi031 ceph-mon[29167]: pgmap v555: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:03.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:03 smithi169 ceph-mon[33112]: pgmap v555: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:03 smithi040 ceph-mon[32945]: pgmap v555: 97 pgs: 97 active+clean; 4.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 -- 172.21.15.31:0/235177759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 msgr2=0x7f9c78102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 --2- 172.21.15.31:0/235177759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78102e40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009900 tx=0x7f9c6c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 -- 172.21.15.31:0/235177759 shutdown_connections 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 --2- 172.21.15.31:0/235177759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c78103380 0x7f9c78109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 --2- 172.21.15.31:0/235177759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78102e40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 --2- 172.21.15.31:0/235177759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c781017d0 0x7f9c78101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 -- 172.21.15.31:0/235177759 >> 172.21.15.31:0/235177759 conn(0x7f9c780fcd60 msgr2=0x7f9c780ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 -- 172.21.15.31:0/235177759 shutdown_connections 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.309+0000 7f9c7e334640 1 -- 172.21.15.31:0/235177759 wait complete. 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 Processor -- start 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 -- start start 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c781017d0 0x7f9c781978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78103380 0x7f9c7819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810d2e0 con 0x7f9c781029c0 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810d160 con 0x7f9c78103380 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7e334640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810d460 con 0x7f9c781017d0 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58304/0 (socket says 172.21.15.31:58304) 2024-04-25T00:37:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c777fe640 1 -- 172.21.15.31:0/3354053727 learned_addr learned my addr 172.21.15.31:0/3354053727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c77fff640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c781017d0 0x7f9c781978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.310+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78103380 0x7f9c7819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c777fe640 1 -- 172.21.15.31:0/3354053727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c781017d0 msgr2=0x7f9c781978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c777fe640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c781017d0 0x7f9c781978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c777fe640 1 -- 172.21.15.31:0/3354053727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78103380 msgr2=0x7f9c7819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c777fe640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78103380 0x7f9c7819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c777fe640 1 -- 172.21.15.31:0/3354053727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7819f4c0 con 0x7f9c781029c0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c777fe640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78197de0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c004770 tx=0x7f9c6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c757fa640 1 -- 172.21.15.31:0/3354053727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c03f030 con 0x7f9c781029c0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c757fa640 1 -- 172.21.15.31:0/3354053727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c6c003cb0 con 0x7f9c781029c0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c757fa640 1 -- 172.21.15.31:0/3354053727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c035070 con 0x7f9c781029c0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7819f750 con 0x7f9c781029c0 2024-04-25T00:37:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.311+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7819fc40 con 0x7f9c781029c0 2024-04-25T00:37:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.313+0000 7f9c757fa640 1 -- 172.21.15.31:0/3354053727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9c6c042030 con 0x7f9c781029c0 2024-04-25T00:37:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 Processor -- start 2024-04-25T00:37:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 -- start start 2024-04-25T00:37:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb1878d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18792340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb18794e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cb186f0cb0 con 0x55cb187929b0 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cb186995b0 con 0x55cb1878d840 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c81043740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb186ecc70 con 0x55cb1878ce50 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb18794e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c7c8d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb1878d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c757fa640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c600a2c30 0x7f9c600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c7c8d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb1878d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35664/0 (socket says 172.21.15.31:35664) 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb18794e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58312/0 (socket says 172.21.15.31:58312) 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2771837991 learned_addr learned my addr 172.21.15.31:0/2771837991 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.314+0000 7f9c77fff640 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18792340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c600a2c30 0x7f9c600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c757fa640 1 -- 172.21.15.31:0/3354053727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9c6c0ef9a0 con 0x7f9c781029c0 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2771837991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 msgr2=0x55cb18792340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18792340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2771837991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 msgr2=0x55cb18794e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb18794e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2771837991 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb18405210 con 0x55cb1878ce50 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb1878d270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c700139d0 tx=0x7f9c70013dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc87881459e82c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c600a2c30 0x7f9c600a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000a000 tx=0x7f9c7001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c5e7fc640 1 -- 172.21.15.31:0/2771837991 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70022070 con 0x55cb1878ce50 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c5e7fc640 1 -- 172.21.15.31:0/2771837991 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c7001cc20 con 0x55cb1878ce50 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.315+0000 7f9c5e7fc640 1 -- 172.21.15.31:0/2771837991 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7001d900 con 0x55cb1878ce50 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 -- 172.21.15.31:0/2771837991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 msgr2=0x55cb1878d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb1878d270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c700139d0 tx=0x7f9c70013dd0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 -- 172.21.15.31:0/2771837991 shutdown_connections 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb18794e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18792340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 --2- 172.21.15.31:0/2771837991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb1878d270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 -- 172.21.15.31:0/2771837991 >> 172.21.15.31:0/2771837991 conn(0x55cb187884c0 msgr2=0x55cb187888c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 -- 172.21.15.31:0/2771837991 shutdown_connections 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 -- 172.21.15.31:0/2771837991 wait complete. 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.316+0000 7f9c81043740 1 -- start start 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 Processor -- start 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb187972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18797810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb1879e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cb18795900 con 0x55cb187929b0 2024-04-25T00:37:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cb186acb50 con 0x55cb1878d840 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb18795a80 con 0x55cb1878ce50 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c81043740 10 client.0 _ll_register_callbacks cb 0x55cb186e7ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb1879e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb1879e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58316/0 (socket says 172.21.15.31:58316) 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2088967635 learned_addr learned my addr 172.21.15.31:0/2088967635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c777fe640 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18797810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 msgr2=0x55cb187972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb187972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2088967635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 msgr2=0x55cb18797810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18797810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.317+0000 7f9c7c8d1640 1 -- 172.21.15.31:0/2088967635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb184d1620 con 0x55cb187929b0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb1879e850 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9c70008010 tx=0x7f9c70005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954c1d311af2e7d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c577fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c577fe640 1 -- 172.21.15.31:0/2088967635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70022070 con 0x55cb187929b0 2024-04-25T00:37:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7001cc20 con 0x55cb187929b0 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c577fe640 1 -- 172.21.15.31:0/2088967635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9c70003b70 con 0x55cb187929b0 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c577fe640 1 -- 172.21.15.31:0/2088967635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c70007520 con 0x55cb187929b0 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c81043740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb187a1c30 con 0x55cb187929b0 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c577fe640 1 -- 172.21.15.31:0/2088967635 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c7001cc20 con 0x55cb187929b0 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c577fe640 1 -- 172.21.15.31:0/2088967635 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c700051c0 con 0x55cb187929b0 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c81043740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.318+0000 7f9c81043740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c81043740 20 client.15909 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c81043740 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb187a3090 0x55cb187a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c81043740 4 client.15909 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb186fe1f0 con 0x55cb187a3090 2024-04-25T00:37:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c77fff640 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb187a3090 0x55cb187a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c77fff640 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb187a3090 0x55cb187a54a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62aa9b1ad2b1129c server_cookie=1e10e659d6c30482 in_seq=0 out_seq=0 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.319+0000 7f9c577fe640 10 client.15909 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c577fe640 1 -- 172.21.15.31:0/2088967635 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb186fe1f0 con 0x55cb187a3090 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c577fe640 10 client.15909 handle_command_reply: tid=0 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 client.15909 shutdown 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 msgr2=0x55cb1879e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb1879e850 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9c70008010 tx=0x7f9c70005f20 comp rx=0 tx=0).stop 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 shutdown_connections 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb187a3090 0x55cb187a54a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb187929b0 0x55cb1879e850 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb18797810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c577fe640 0 client.15909 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 --2- 172.21.15.31:0/2088967635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878ce50 0x55cb187972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 >> 172.21.15.31:0/2088967635 conn(0x55cb187884c0 msgr2=0x55cb187875c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.320+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 shutdown_connections 2024-04-25T00:37:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.321+0000 7f9c81043740 1 -- 172.21.15.31:0/2088967635 wait complete. 2024-04-25T00:37:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.321+0000 7f9c81043740 20 client.15909 trim_cache size 0 max 16384 2024-04-25T00:37:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.335+0000 7f9c81043740 1 Processor -- start 2024-04-25T00:37:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.335+0000 7f9c81043740 1 -- start start 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c81043740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb1878b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c81043740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb1878be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c81043740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb187929b0 0x55cb187a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c81043740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781a8af0 con 0x55cb1878ce50 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c81043740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781aa630 con 0x55cb1878d840 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c81043740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781a91c0 con 0x55cb187929b0 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb1878b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb1878b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58320/0 (socket says 172.21.15.31:58320) 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 -- 172.21.15.31:0/738187674 learned_addr learned my addr 172.21.15.31:0/738187674 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 -- 172.21.15.31:0/738187674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb187929b0 msgr2=0x55cb187a9da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb187929b0 0x55cb187a9da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 -- 172.21.15.31:0/738187674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 msgr2=0x55cb1878be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb1878be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 -- 172.21.15.31:0/738187674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb187a5b20 con 0x55cb1878ce50 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c77fff640 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb1878b950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9c68004590 tx=0x7f9c6800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a87b3431c2902a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c5dffb640 1 -- 172.21.15.31:0/738187674 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68003ea0 con 0x55cb1878ce50 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c5dffb640 1 -- 172.21.15.31:0/738187674 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c6801c8d0 con 0x55cb1878ce50 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.336+0000 7f9c5dffb640 1 -- 172.21.15.31:0/738187674 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68010430 con 0x55cb1878ce50 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 -- 172.21.15.31:0/738187674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 msgr2=0x55cb1878b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb1878b950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9c68004590 tx=0x7f9c6800b630 comp rx=0 tx=0).stop 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 -- 172.21.15.31:0/738187674 shutdown_connections 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb187929b0 0x55cb187a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb1878d840 0x55cb1878be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 --2- 172.21.15.31:0/738187674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb1878b950 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 -- 172.21.15.31:0/738187674 >> 172.21.15.31:0/738187674 conn(0x55cb187884c0 msgr2=0x55cb1878b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 -- 172.21.15.31:0/738187674 shutdown_connections 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 -- 172.21.15.31:0/738187674 wait complete. 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.337+0000 7f9c81043740 1 -- start start 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 Processor -- start 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb188047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878d840 0x55cb18804d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb187929b0 0x55cb1880bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781a9f60 con 0x55cb1878ce50 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781a9890 con 0x55cb187929b0 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781a6ad0 con 0x55cb1878d840 2024-04-25T00:37:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c81043740 10 client.0 _ll_register_callbacks cb 0x7f9c70020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb188047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb188047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58334/0 (socket says 172.21.15.31:58334) 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 -- 172.21.15.31:0/1593350215 learned_addr learned my addr 172.21.15.31:0/1593350215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c7c8d1640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb187929b0 0x55cb1880bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c777fe640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878d840 0x55cb18804d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 -- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878d840 msgr2=0x55cb18804d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878d840 0x55cb18804d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 -- 172.21.15.31:0/1593350215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb187929b0 msgr2=0x55cb1880bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb187929b0 0x55cb1880bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 -- 172.21.15.31:0/1593350215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1880c330 con 0x55cb1878ce50 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c77fff640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb188047c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9c68003ca0 tx=0x7f9c68003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4c085d8a45a7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.338+0000 7f9c577fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c577fe640 1 -- 172.21.15.31:0/1593350215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6801c620 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c577fe640 1 -- 172.21.15.31:0/1593350215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9c68011440 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c6801c8d0 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c577fe640 1 -- 172.21.15.31:0/1593350215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c68011640 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 10 client.15915 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb1880d640 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c577fe640 1 -- 172.21.15.31:0/1593350215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c6801c8d0 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c577fe640 1 -- 172.21.15.31:0/1593350215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c6802c070 con 0x55cb1878ce50 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 20 client.15915 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb1880e040 0x55cb18810460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 4 client.15915 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb186f8540 con 0x55cb1880e040 2024-04-25T00:37:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.339+0000 7f9c777fe640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb1880e040 0x55cb18810460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.340+0000 7f9c577fe640 10 client.15915 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.340+0000 7f9c777fe640 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb1880e040 0x55cb18810460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57e2d50a7adee2 server_cookie=888162392aabe70a in_seq=0 out_seq=0 2024-04-25T00:37:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c577fe640 1 -- 172.21.15.31:0/1593350215 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb186f8540 con 0x55cb1880e040 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c577fe640 10 client.15915 handle_command_reply: tid=0 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 client.15915 shutdown 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 msgr2=0x55cb188047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb188047c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9c68003ca0 tx=0x7f9c68003cd0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 shutdown_connections 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55cb1880e040 0x55cb18810460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb187929b0 0x55cb1880bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb1878d840 0x55cb18804d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c577fe640 0 client.15915 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 --2- 172.21.15.31:0/1593350215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cb1878ce50 0x55cb188047c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 >> 172.21.15.31:0/1593350215 conn(0x55cb187884c0 msgr2=0x55cb1878bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 shutdown_connections 2024-04-25T00:37:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 1 -- 172.21.15.31:0/1593350215 wait complete. 2024-04-25T00:37:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c81043740 20 client.15915 trim_cache size 0 max 16384 2024-04-25T00:37:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:37:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:37:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:37:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c600a2c30 msgr2=0x7f9c600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c600a2c30 0x7f9c600a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000a000 tx=0x7f9c7001d000 comp rx=0 tx=0).stop 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 msgr2=0x7f9c78197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78197de0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c004770 tx=0x7f9c6c0042f0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 shutdown_connections 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78103380 0x7f9c7819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9c600a2c30 0x7f9c600a50f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c781029c0 0x7f9c78197de0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 --2- 172.21.15.31:0/3354053727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c781017d0 0x7f9c781978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 >> 172.21.15.31:0/3354053727 conn(0x7f9c780fcd60 msgr2=0x7f9c78105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 shutdown_connections 2024-04-25T00:37:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:04.358+0000 7f9c7e334640 1 -- 172.21.15.31:0/3354053727 wait complete. 2024-04-25T00:37:05.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:05 smithi031 ceph-mon[29167]: pgmap v556: 97 pgs: 97 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:37:05.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:05 smithi169 ceph-mon[33112]: pgmap v556: 97 pgs: 97 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:37:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:05 smithi040 ceph-mon[32945]: pgmap v556: 97 pgs: 97 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:37:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:07 smithi031 ceph-mon[29167]: pgmap v557: 97 pgs: 97 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:37:07.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:07 smithi169 ceph-mon[33112]: pgmap v557: 97 pgs: 97 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:37:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:07 smithi040 ceph-mon[32945]: pgmap v557: 97 pgs: 97 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:37:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:09 smithi031 ceph-mon[29167]: pgmap v558: 97 pgs: 97 active+clean; 4.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:37:09.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:09 smithi169 ceph-mon[33112]: pgmap v558: 97 pgs: 97 active+clean; 4.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:37:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:09 smithi040 ceph-mon[32945]: pgmap v558: 97 pgs: 97 active+clean; 4.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:37:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:11 smithi031 ceph-mon[29167]: pgmap v559: 97 pgs: 97 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:11.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:11 smithi169 ceph-mon[33112]: pgmap v559: 97 pgs: 97 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:11 smithi040 ceph-mon[32945]: pgmap v559: 97 pgs: 97 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:13 smithi031 ceph-mon[29167]: pgmap v560: 97 pgs: 97 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:13 smithi169 ceph-mon[33112]: pgmap v560: 97 pgs: 97 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:13 smithi040 ceph-mon[32945]: pgmap v560: 97 pgs: 97 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:15 smithi031 ceph-mon[29167]: pgmap v561: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:37:15.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:15 smithi169 ceph-mon[33112]: pgmap v561: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:37:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:15 smithi040 ceph-mon[32945]: pgmap v561: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:37:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:17 smithi169 ceph-mon[33112]: pgmap v562: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:17.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:17 smithi040 ceph-mon[32945]: pgmap v562: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2879303865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 msgr2=0x7fdf14106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2879303865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14106fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009900 tx=0x7fdf0802a410 comp rx=0 tx=0).stop 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2879303865 shutdown_connections 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2879303865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141074e0 0x7fdf141098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2879303865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14106fa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2879303865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf14102280 0x7fdf14104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2879303865 >> 172.21.15.31:0/2879303865 conn(0x7fdf140fbc10 msgr2=0x7fdf140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2879303865 shutdown_connections 2024-04-25T00:37:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.458+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2879303865 wait complete. 2024-04-25T00:37:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 Processor -- start 2024-04-25T00:37:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 -- start start 2024-04-25T00:37:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf14102280 0x7fdf141979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf141074e0 0x7fdf1419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410d2a0 con 0x7fdf14104bb0 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410d120 con 0x7fdf141074e0 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf1aa3f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410d420 con 0x7fdf14102280 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44792/0 (socket says 172.21.15.31:44792) 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.459+0000 7fdf137fe640 1 -- 172.21.15.31:0/2216750619 learned_addr learned my addr 172.21.15.31:0/2216750619 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf141074e0 0x7fdf1419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf14102280 0x7fdf141979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf137fe640 1 -- 172.21.15.31:0/2216750619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf14102280 msgr2=0x7fdf141979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf14102280 0x7fdf141979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf137fe640 1 -- 172.21.15.31:0/2216750619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf141074e0 msgr2=0x7fdf1419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf141074e0 0x7fdf1419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf137fe640 1 -- 172.21.15.31:0/2216750619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1419f5c0 con 0x7fdf14104bb0 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14197ee0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdf080314e0 tx=0x7fdf080042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf117fa640 1 -- 172.21.15.31:0/2216750619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0803f030 con 0x7fdf14104bb0 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1419f850 con 0x7fdf14104bb0 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf117fa640 1 -- 172.21.15.31:0/2216750619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf08003cb0 con 0x7fdf14104bb0 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf117fa640 1 -- 172.21.15.31:0/2216750619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08035070 con 0x7fdf14104bb0 2024-04-25T00:37:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.460+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1419fd40 con 0x7fdf14104bb0 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.463+0000 7fdf1d82e740 1 Processor -- start 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.463+0000 7fdf117fa640 1 -- 172.21.15.31:0/2216750619 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdf08042030 con 0x7fdf14104bb0 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.463+0000 7fdf1d82e740 1 -- start start 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185592e50 0x55f185593270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185598340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f18559ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f1854f6cb0 con 0x55f185593840 2024-04-25T00:37:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f18549f5b0 con 0x55f185592e50 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f1854f2c70 con 0x55f1855989b0 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185598340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185598340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44794/0 (socket says 172.21.15.31:44794) 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 -- 172.21.15.31:0/2934458245 learned_addr learned my addr 172.21.15.31:0/2934458245 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f18559ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185592e50 0x55f185593270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 -- 172.21.15.31:0/2934458245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 msgr2=0x55f18559ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f18559ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 -- 172.21.15.31:0/2934458245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185592e50 msgr2=0x55f185593270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185592e50 0x55f185593270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 -- 172.21.15.31:0/2934458245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18520b210 con 0x55f185593840 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185592e50 0x55f185593270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf117fa640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdefc0a2c30 0x7fdefc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185598340 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400bd10 tx=0x7fdf040107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f9ed1c00969275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.464+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdefc0a2c30 0x7fdefc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf117fa640 1 -- 172.21.15.31:0/2216750619 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fdf080ef5e0 con 0x7fdf14104bb0 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdefa7fc640 1 -- 172.21.15.31:0/2934458245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004030 con 0x55f185593840 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdefa7fc640 1 -- 172.21.15.31:0/2934458245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf040035f0 con 0x55f185593840 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdefa7fc640 1 -- 172.21.15.31:0/2934458245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf040117b0 con 0x55f185593840 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2934458245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 msgr2=0x55f185598340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdefc0a2c30 0x7fdefc0a50f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f1855940a0 tx=0x7fdf0c0096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185598340 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400bd10 tx=0x7fdf040107a0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2934458245 shutdown_connections 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f18559ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185598340 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2934458245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185592e50 0x55f185593270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2934458245 >> 172.21.15.31:0/2934458245 conn(0x55f18558e4c0 msgr2=0x55f18558e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2934458245 shutdown_connections 2024-04-25T00:37:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.465+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2934458245 wait complete. 2024-04-25T00:37:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 -- start start 2024-04-25T00:37:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 Processor -- start 2024-04-25T00:37:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f18559d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185593840 0x55f18559d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f1855a4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f18559b960 con 0x55f185592e50 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f1854b2b50 con 0x55f1855989b0 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f18559bae0 con 0x55f185593840 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf1d82e740 10 client.0 _ll_register_callbacks cb 0x55f1854edca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f18559d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f18559d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44796/0 (socket says 172.21.15.31:44796) 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.466+0000 7fdf13fff640 1 -- 172.21.15.31:0/2988995509 learned_addr learned my addr 172.21.15.31:0/2988995509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f1855a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185593840 0x55f18559d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf13fff640 1 -- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185593840 msgr2=0x55f18559d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185593840 0x55f18559d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf13fff640 1 -- 172.21.15.31:0/2988995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 msgr2=0x55f1855a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f1855a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf13fff640 1 -- 172.21.15.31:0/2988995509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1852d7620 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185593840 0x55f18559d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf13fff640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f18559d2a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000c00 tx=0x7fdf040058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999de721b4ed9ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdef37fe640 1 -- 172.21.15.31:0/2988995509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04029060 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf040035f0 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdef37fe640 1 -- 172.21.15.31:0/2988995509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fdf04003fa0 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdef37fe640 1 -- 172.21.15.31:0/2988995509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf040041c0 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf1d82e740 10 client.15927 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.467+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1855a7b50 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdef37fe640 1 -- 172.21.15.31:0/2988995509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf040035f0 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdef37fe640 1 -- 172.21.15.31:0/2988995509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdf04004800 con 0x55f185592e50 2024-04-25T00:37:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf1d82e740 10 client.15927 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf1d82e740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf1d82e740 20 client.15927 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f1855a8f60 0x55f1855ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf1d82e740 4 client.15927 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1855041f0 con 0x55f1855a8f60 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.468+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f1855a8f60 0x55f1855ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdef37fe640 10 client.15927 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdf137fe640 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f1855a8f60 0x55f1855ab350 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e879035c9fb70ed5 server_cookie=34115644ec28f7a7 in_seq=0 out_seq=0 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdef37fe640 1 -- 172.21.15.31:0/2988995509 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1855041f0 con 0x55f1855a8f60 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdef37fe640 10 client.15927 handle_command_reply: tid=0 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdf1d82e740 1 client.15927 shutdown 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 msgr2=0x55f18559d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.469+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f18559d2a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000c00 tx=0x7fdf040058e0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 shutdown_connections 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f1855a8f60 0x55f1855ab350 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f1855a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185593840 0x55f18559d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdef37fe640 0 client.15927 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/2988995509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f18559d2a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 >> 172.21.15.31:0/2988995509 conn(0x55f18558e4c0 msgr2=0x55f18558d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 shutdown_connections 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 1 -- 172.21.15.31:0/2988995509 wait complete. 2024-04-25T00:37:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.470+0000 7fdf1d82e740 20 client.15927 trim_cache size 0 max 16384 2024-04-25T00:37:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 Processor -- start 2024-04-25T00:37:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 -- start start 2024-04-25T00:37:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185592e50 0x55f185591550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185591a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f18559c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141a8c60 con 0x55f185593840 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141aa7a0 con 0x55f1855989b0 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141a9330 con 0x55f185592e50 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185591a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185591a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44800/0 (socket says 172.21.15.31:44800) 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 -- 172.21.15.31:0/1936549447 learned_addr learned my addr 172.21.15.31:0/1936549447 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf13fff640 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185592e50 0x55f185591550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f18559c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 -- 172.21.15.31:0/1936549447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185592e50 msgr2=0x55f185591550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185592e50 0x55f185591550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 -- 172.21.15.31:0/1936549447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 msgr2=0x55f18559c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f18559c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf137fe640 1 -- 172.21.15.31:0/1936549447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18559bc10 con 0x55f185593840 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.485+0000 7fdf13fff640 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185592e50 0x55f185591550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf137fe640 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185591a90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdf080b8890 tx=0x7fdf080ecf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ab162137cb5c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdef9ffb640 1 -- 172.21.15.31:0/1936549447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08049040 con 0x55f185593840 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdef9ffb640 1 -- 172.21.15.31:0/1936549447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf080bbec0 con 0x55f185593840 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdef9ffb640 1 -- 172.21.15.31:0/1936549447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080b8530 con 0x55f185593840 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 -- 172.21.15.31:0/1936549447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 msgr2=0x55f185591a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185591a90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdf080b8890 tx=0x7fdf080ecf80 comp rx=0 tx=0).stop 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 -- 172.21.15.31:0/1936549447 shutdown_connections 2024-04-25T00:37:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1855989b0 0x55f18559c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185593840 0x55f185591a90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/1936549447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f185592e50 0x55f185591550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 -- 172.21.15.31:0/1936549447 >> 172.21.15.31:0/1936549447 conn(0x55f18558e4c0 msgr2=0x55f185594ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 -- 172.21.15.31:0/1936549447 shutdown_connections 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.486+0000 7fdf1d82e740 1 -- 172.21.15.31:0/1936549447 wait complete. 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 -- start start 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 Processor -- start 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f1855a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185593840 0x55f1855a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f1855a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141aa0d0 con 0x55f185592e50 2024-04-25T00:37:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141a9a00 con 0x55f185593840 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141a6c40 con 0x55f1855989b0 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf1d82e740 10 client.0 _ll_register_callbacks cb 0x7fdf080f5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f1855a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f1855a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44812/0 (socket says 172.21.15.31:44812) 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf13fff640 1 -- 172.21.15.31:0/841711167 learned_addr learned my addr 172.21.15.31:0/841711167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf137fe640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185593840 0x55f1855a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.487+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f1855a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf13fff640 1 -- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 msgr2=0x55f1855a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf13fff640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f1855a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf13fff640 1 -- 172.21.15.31:0/841711167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185593840 msgr2=0x55f1855a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf13fff640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185593840 0x55f1855a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf13fff640 1 -- 172.21.15.31:0/841711167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1855a2b80 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf18fdc640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f1855a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf13fff640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f1855a1090 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdf040107e0 tx=0x7fdf04003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b648cd3bf5aae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdef37fe640 1 -- 172.21.15.31:0/841711167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04029050 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdef37fe640 1 -- 172.21.15.31:0/841711167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fdf04011820 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf080bbec0 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdef37fe640 1 -- 172.21.15.31:0/841711167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf0403f050 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf1d82e740 10 client.15933 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1855a3e90 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdef37fe640 1 -- 172.21.15.31:0/841711167 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf080bbec0 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.488+0000 7fdef37fe640 1 -- 172.21.15.31:0/841711167 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdf040047b0 con 0x55f185592e50 2024-04-25T00:37:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf1d82e740 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf1d82e740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf1d82e740 20 client.15933 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f18560f920 0x55f185611d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf1d82e740 4 client.15933 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1854fe540 con 0x55f18560f920 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf137fe640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f18560f920 0x55f185611d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdf137fe640 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f18560f920 0x55f185611d40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876f8f683ca40287 server_cookie=4c9d48a7a1a60c1a in_seq=0 out_seq=0 2024-04-25T00:37:17.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.489+0000 7fdef37fe640 10 client.15933 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdef37fe640 1 -- 172.21.15.31:0/841711167 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1854fe540 con 0x55f18560f920 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdef37fe640 10 client.15933 handle_command_reply: tid=0 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 client.15933 shutdown 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 msgr2=0x55f1855a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f1855a1090 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdf040107e0 tx=0x7fdf04003820 comp rx=0 tx=0).stop 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 shutdown_connections 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f18560f920 0x55f185611d40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1855989b0 0x55f1855a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f185593840 0x55f1855a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdef37fe640 0 client.15933 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 --2- 172.21.15.31:0/841711167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f185592e50 0x55f1855a1090 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 >> 172.21.15.31:0/841711167 conn(0x55f18558e4c0 msgr2=0x55f185591550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 shutdown_connections 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 1 -- 172.21.15.31:0/841711167 wait complete. 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.506+0000 7fdf1d82e740 20 client.15933 trim_cache size 0 max 16384 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:37:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:37:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:37:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:37:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdefc0a2c30 msgr2=0x7fdefc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdefc0a2c30 0x7fdefc0a50f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f1855940a0 tx=0x7fdf0c0096a0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 msgr2=0x7fdf14197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14197ee0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdf080314e0 tx=0x7fdf080042e0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 shutdown_connections 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf141074e0 0x7fdf1419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fdefc0a2c30 0x7fdefc0a50f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf14104bb0 0x7fdf14197ee0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 --2- 172.21.15.31:0/2216750619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf14102280 0x7fdf141979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 >> 172.21.15.31:0/2216750619 conn(0x7fdf140fbc10 msgr2=0x7fdf14102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 shutdown_connections 2024-04-25T00:37:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:17.507+0000 7fdf1aa3f640 1 -- 172.21.15.31:0/2216750619 wait complete. 2024-04-25T00:37:17.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:17 smithi031 ceph-mon[29167]: pgmap v562: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:19.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:19 smithi169 ceph-mon[33112]: pgmap v563: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:19.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:19 smithi040 ceph-mon[32945]: pgmap v563: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:19.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:19 smithi031 ceph-mon[29167]: pgmap v563: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:21.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:21 smithi169 ceph-mon[33112]: pgmap v564: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-25T00:37:21.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:21 smithi040 ceph-mon[32945]: pgmap v564: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-25T00:37:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:21 smithi031 ceph-mon[29167]: pgmap v564: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-25T00:37:23.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:23 smithi169 ceph-mon[33112]: pgmap v565: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:23.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:23 smithi040 ceph-mon[32945]: pgmap v565: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:23.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:23 smithi031 ceph-mon[29167]: pgmap v565: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:25.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:25 smithi169 ceph-mon[33112]: pgmap v566: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:25.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:25 smithi040 ceph-mon[32945]: pgmap v566: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:25.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:25 smithi031 ceph-mon[29167]: pgmap v566: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:27.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:27 smithi169 ceph-mon[33112]: pgmap v567: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:27.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:27 smithi040 ceph-mon[32945]: pgmap v567: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:27.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:27 smithi031 ceph-mon[29167]: pgmap v567: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:37:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:29 smithi169 ceph-mon[33112]: pgmap v568: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:29.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:29 smithi040 ceph-mon[32945]: pgmap v568: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:29 smithi031 ceph-mon[29167]: pgmap v568: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 -- 172.21.15.31:0/1276671140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35640ffa90 msgr2=0x7f35640ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 --2- 172.21.15.31:0/1276671140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35640ffa90 0x7f35640ffeb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3560009930 tx=0x7f356002a420 comp rx=0 tx=0).stop 2024-04-25T00:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 -- 172.21.15.31:0/1276671140 shutdown_connections 2024-04-25T00:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 --2- 172.21.15.31:0/1276671140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3564103420 0x7f3564109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 --2- 172.21.15.31:0/1276671140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3564102a60 0x7f3564102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 --2- 172.21.15.31:0/1276671140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35640ffa90 0x7f35640ffeb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 -- 172.21.15.31:0/1276671140 >> 172.21.15.31:0/1276671140 conn(0x7f35640fb3c0 msgr2=0x7f35640fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 -- 172.21.15.31:0/1276671140 shutdown_connections 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.606+0000 7f356b4a5640 1 -- 172.21.15.31:0/1276671140 wait complete. 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 Processor -- start 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 -- start start 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35640ffa90 0x7f35641979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 0x7f3564197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3564103420 0x7f356419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f356410d2f0 con 0x7f3564102a60 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f356410d170 con 0x7f3564103420 2024-04-25T00:37:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f356b4a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f356410d470 con 0x7f35640ffa90 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 0x7f3564197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 0x7f3564197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37282/0 (socket says 172.21.15.31:37282) 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 -- 172.21.15.31:0/3838762577 learned_addr learned my addr 172.21.15.31:0/3838762577 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3569a42640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3564103420 0x7f356419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3569241640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35640ffa90 0x7f35641979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 -- 172.21.15.31:0/3838762577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35640ffa90 msgr2=0x7f35641979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35640ffa90 0x7f35641979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 -- 172.21.15.31:0/3838762577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3564103420 msgr2=0x7f356419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3564103420 0x7f356419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.607+0000 7f3568a40640 1 -- 172.21.15.31:0/3838762577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356419f5c0 con 0x7f3564102a60 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f3569241640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35640ffa90 0x7f35641979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f3568a40640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 0x7f3564197ee0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f355400d950 tx=0x7f355400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f355a7fc640 1 -- 172.21.15.31:0/3838762577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554017070 con 0x7f3564102a60 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356419f8b0 con 0x7f3564102a60 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f355a7fc640 1 -- 172.21.15.31:0/3838762577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3554009c30 con 0x7f3564102a60 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f355a7fc640 1 -- 172.21.15.31:0/3838762577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400f620 con 0x7f3564102a60 2024-04-25T00:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.608+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356419fe50 con 0x7f3564102a60 2024-04-25T00:37:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.610+0000 7f355a7fc640 1 -- 172.21.15.31:0/3838762577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f35540109a0 con 0x7f3564102a60 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 Processor -- start 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 -- start start 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207503e50 0x55c207504270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 0x55c207509340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c20750be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c207467cb0 con 0x55c207504840 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c2074105b0 con 0x55c207503e50 2024-04-25T00:37:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f356e1b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c207463c70 con 0x55c2075099b0 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 0x55c207509340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f355a7fc640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35500a2c30 0x7f35500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 0x55c207509340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37288/0 (socket says 172.21.15.31:37288) 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f3569241640 1 -- 172.21.15.31:0/3487946851 learned_addr learned my addr 172.21.15.31:0/3487946851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f3569a42640 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207503e50 0x55c207504270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f3569a42640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35500a2c30 0x7f35500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.611+0000 7f3568a40640 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c20750be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569241640 1 -- 172.21.15.31:0/3487946851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 msgr2=0x55c20750be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569241640 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c20750be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569241640 1 -- 172.21.15.31:0/3487946851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207503e50 msgr2=0x55c207504270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569241640 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207503e50 0x55c207504270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569241640 1 -- 172.21.15.31:0/3487946851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20717c210 con 0x55c207504840 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f355a7fc640 1 -- 172.21.15.31:0/3838762577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f35540ced20 con 0x7f3564102a60 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3568a40640 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c20750be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569241640 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 0x55c207509340 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3560000c00 tx=0x7f35600316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97c939e1d9036e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f3569a42640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35500a2c30 0x7f35500a50f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55c2075050a0 tx=0x7f355c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f354f7fe640 1 -- 172.21.15.31:0/3487946851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560035030 con 0x55c207504840 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f354f7fe640 1 -- 172.21.15.31:0/3487946851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3560031cb0 con 0x55c207504840 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.612+0000 7f354f7fe640 1 -- 172.21.15.31:0/3487946851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600046c0 con 0x55c207504840 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 -- 172.21.15.31:0/3487946851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 msgr2=0x55c207509340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 0x55c207509340 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3560000c00 tx=0x7f35600316c0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 -- 172.21.15.31:0/3487946851 shutdown_connections 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c20750be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207504840 0x55c207509340 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3487946851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207503e50 0x55c207504270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 -- 172.21.15.31:0/3487946851 >> 172.21.15.31:0/3487946851 conn(0x55c2074ff4c0 msgr2=0x55c2074ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 -- 172.21.15.31:0/3487946851 shutdown_connections 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 -- 172.21.15.31:0/3487946851 wait complete. 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 -- start start 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.613+0000 7f356e1b3740 1 Processor -- start 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c20750e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c207504840 0x55c20750e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2075099b0 0x55c2075157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c20750c900 con 0x55c207503e50 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c207423b50 con 0x55c2075099b0 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c20750ca80 con 0x55c207504840 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f356e1b3740 10 client.0 _ll_register_callbacks cb 0x55c20745eca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c20750e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c20750e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37300/0 (socket says 172.21.15.31:37300) 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 -- 172.21.15.31:0/3993824156 learned_addr learned my addr 172.21.15.31:0/3993824156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3568a40640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c207504840 0x55c20750e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569a42640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2075099b0 0x55c2075157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 -- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c207504840 msgr2=0x55c20750e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c207504840 0x55c20750e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 -- 172.21.15.31:0/3993824156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2075099b0 msgr2=0x55c2075157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2075099b0 0x55c2075157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569241640 1 -- 172.21.15.31:0/3993824156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c207248620 con 0x55c207503e50 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.614+0000 7f3569a42640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2075099b0 0x55c2075157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f3569241640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c20750e220 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3560006010 tx=0x7f35600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb198c0ec83b988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f354cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f354cff9640 1 -- 172.21.15.31:0/3993824156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560035030 con 0x55c207503e50 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3560031cb0 con 0x55c207503e50 2024-04-25T00:37:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f354cff9640 1 -- 172.21.15.31:0/3993824156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3560004260 con 0x55c207503e50 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f354cff9640 1 -- 172.21.15.31:0/3993824156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3560004460 con 0x55c207503e50 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 10 client.15945 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c207518a80 con 0x55c207503e50 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f354cff9640 1 -- 172.21.15.31:0/3993824156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3560031cb0 con 0x55c207503e50 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f354cff9640 1 -- 172.21.15.31:0/3993824156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3560056070 con 0x55c207503e50 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 10 client.15945 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 20 client.15945 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c207519dd0 0x55c20751c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 4 client.15945 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.615+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2074751f0 con 0x55c207519dd0 2024-04-25T00:37:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.616+0000 7f3568a40640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c207519dd0 0x55c20751c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.616+0000 7f3568a40640 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c207519dd0 0x55c20751c1c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab82545e44db3a49 server_cookie=9932e07b51967ee3 in_seq=0 out_seq=0 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.616+0000 7f354cff9640 10 client.15945 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f354cff9640 1 -- 172.21.15.31:0/3993824156 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2074751f0 con 0x55c207519dd0 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f354cff9640 10 client.15945 handle_command_reply: tid=0 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 client.15945 shutdown 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 msgr2=0x55c20750e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c20750e220 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3560006010 tx=0x7f35600048f0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 shutdown_connections 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c207519dd0 0x55c20751c1c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2075099b0 0x55c2075157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f354cff9640 0 client.15945 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c207504840 0x55c20750e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 --2- 172.21.15.31:0/3993824156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c20750e220 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 >> 172.21.15.31:0/3993824156 conn(0x55c2074ff4c0 msgr2=0x55c2074fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 shutdown_connections 2024-04-25T00:37:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 1 -- 172.21.15.31:0/3993824156 wait complete. 2024-04-25T00:37:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.617+0000 7f356e1b3740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T00:37:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.632+0000 7f356e1b3740 1 Processor -- start 2024-04-25T00:37:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.632+0000 7f356e1b3740 1 -- start start 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207502cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207515760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c2074fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f356e1b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f35641a8b90 con 0x55c207503e50 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f356e1b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35641aa6d0 con 0x55c207504840 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f356e1b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35641a9260 con 0x55c2075099b0 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207502cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207502cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37308/0 (socket says 172.21.15.31:37308) 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 -- 172.21.15.31:0/4118595932 learned_addr learned my addr 172.21.15.31:0/4118595932 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3568a40640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207515760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569a42640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c2074fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 -- 172.21.15.31:0/4118595932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 msgr2=0x55c2074fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c2074fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 -- 172.21.15.31:0/4118595932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 msgr2=0x55c207515760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207515760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 -- 172.21.15.31:0/4118595932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c207500230 con 0x55c207503e50 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3568a40640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207515760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569a42640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c2074fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f3569241640 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207502cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f356002a940 tx=0x7f35600316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce6778d29ce11d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f354effd640 1 -- 172.21.15.31:0/4118595932 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560035040 con 0x55c207503e50 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f354effd640 1 -- 172.21.15.31:0/4118595932 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3560053070 con 0x55c207503e50 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.633+0000 7f354effd640 1 -- 172.21.15.31:0/4118595932 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560004e80 con 0x55c207503e50 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 -- 172.21.15.31:0/4118595932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 msgr2=0x55c207502cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207502cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f356002a940 tx=0x7f35600316c0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 -- 172.21.15.31:0/4118595932 shutdown_connections 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c2074fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207515760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 --2- 172.21.15.31:0/4118595932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207502cd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 -- 172.21.15.31:0/4118595932 >> 172.21.15.31:0/4118595932 conn(0x55c2074ff4c0 msgr2=0x55c2074fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 -- 172.21.15.31:0/4118595932 shutdown_connections 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 -- 172.21.15.31:0/4118595932 wait complete. 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 -- start start 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.634+0000 7f356e1b3740 1 Processor -- start 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207512d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207513280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c207514240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f35641aa000 con 0x55c207503e50 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35641a9930 con 0x55c207504840 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35641a6b70 con 0x55c2075099b0 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f356e1b3740 10 client.0 _ll_register_callbacks cb 0x7f3560031610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207512d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207512d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37320/0 (socket says 172.21.15.31:37320) 2024-04-25T00:37:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 -- 172.21.15.31:0/1886252188 learned_addr learned my addr 172.21.15.31:0/1886252188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569a42640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c207514240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 -- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 msgr2=0x55c207514240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c207514240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 -- 172.21.15.31:0/1886252188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 msgr2=0x55c207513280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3568a40640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207513280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207513280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 -- 172.21.15.31:0/1886252188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c207514830 con 0x55c207503e50 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3568a40640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207513280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f3569241640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207512d40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3560003c50 tx=0x7f3560003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901b6ba9b0073adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.635+0000 7f354cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f354cff9640 1 -- 172.21.15.31:0/1886252188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356005c030 con 0x55c207503e50 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f354cff9640 1 -- 172.21.15.31:0/1886252188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3560049040 con 0x55c207503e50 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3560053070 con 0x55c207503e50 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f354cff9640 1 -- 172.21.15.31:0/1886252188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3560035070 con 0x55c207503e50 2024-04-25T00:37:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 10 client.15951 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c207515ab0 con 0x55c207503e50 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f354cff9640 1 -- 172.21.15.31:0/1886252188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3560053070 con 0x55c207503e50 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f354cff9640 1 -- 172.21.15.31:0/1886252188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3560056030 con 0x55c207503e50 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 10 client.15951 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 20 client.15951 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2075808a0 0x55c207582cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 4 client.15951 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.636+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c20746f540 con 0x55c2075808a0 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.637+0000 7f3568a40640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2075808a0 0x55c207582cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.637+0000 7f3568a40640 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2075808a0 0x55c207582cc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca2c258f0831b1f server_cookie=d9809d9d2fd8979f in_seq=0 out_seq=0 2024-04-25T00:37:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.637+0000 7f354cff9640 10 client.15951 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f354cff9640 1 -- 172.21.15.31:0/1886252188 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c20746f540 con 0x55c2075808a0 2024-04-25T00:37:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f354cff9640 10 client.15951 handle_command_reply: tid=0 2024-04-25T00:37:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 client.15951 shutdown 2024-04-25T00:37:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 msgr2=0x55c207512d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207512d40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3560003c50 tx=0x7f3560003820 comp rx=0 tx=0).stop 2024-04-25T00:37:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 shutdown_connections 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2075808a0 0x55c207582cc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2075099b0 0x55c207514240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c207504840 0x55c207513280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f354cff9640 0 client.15951 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 --2- 172.21.15.31:0/1886252188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c207503e50 0x55c207512d40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 >> 172.21.15.31:0/1886252188 conn(0x55c2074ff4c0 msgr2=0x55c207502b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 shutdown_connections 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 1 -- 172.21.15.31:0/1886252188 wait complete. 2024-04-25T00:37:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.653+0000 7f356e1b3740 20 client.15951 trim_cache size 0 max 16384 2024-04-25T00:37:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:37:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:37:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:37:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35500a2c30 msgr2=0x7f35500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35500a2c30 0x7f35500a50f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55c2075050a0 tx=0x7f355c009600 comp rx=0 tx=0).stop 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 msgr2=0x7f3564197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 0x7f3564197ee0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f355400d950 tx=0x7f355400de10 comp rx=0 tx=0).stop 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 shutdown_connections 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3564103420 0x7f356419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f35500a2c30 0x7f35500a50f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3564102a60 0x7f3564197ee0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 --2- 172.21.15.31:0/3838762577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35640ffa90 0x7f35641979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 >> 172.21.15.31:0/3838762577 conn(0x7f35640fb3c0 msgr2=0x7f3564105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 shutdown_connections 2024-04-25T00:37:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:30.654+0000 7f356b4a5640 1 -- 172.21.15.31:0/3838762577 wait complete. 2024-04-25T00:37:31.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:31 smithi169 ceph-mon[33112]: pgmap v569: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:31 smithi040 ceph-mon[32945]: pgmap v569: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:31 smithi031 ceph-mon[29167]: pgmap v569: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:33.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:33 smithi169 ceph-mon[33112]: pgmap v570: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:37:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:33 smithi040 ceph-mon[32945]: pgmap v570: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:37:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:33 smithi031 ceph-mon[29167]: pgmap v570: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:37:35.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:35 smithi169 ceph-mon[33112]: pgmap v571: 97 pgs: 97 active+clean; 645 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 23 op/s 2024-04-25T00:37:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:35 smithi040 ceph-mon[32945]: pgmap v571: 97 pgs: 97 active+clean; 645 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 23 op/s 2024-04-25T00:37:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:35 smithi031 ceph-mon[29167]: pgmap v571: 97 pgs: 97 active+clean; 645 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 23 op/s 2024-04-25T00:37:37.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:37 smithi169 ceph-mon[33112]: pgmap v572: 97 pgs: 97 active+clean; 645 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:37:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:37 smithi040 ceph-mon[32945]: pgmap v572: 97 pgs: 97 active+clean; 645 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:37:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:37 smithi031 ceph-mon[29167]: pgmap v572: 97 pgs: 97 active+clean; 645 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:37:39.324 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:39 smithi031 ceph-mon[29167]: pgmap v573: 97 pgs: 97 active+clean; 698 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:39 smithi169 ceph-mon[33112]: pgmap v573: 97 pgs: 97 active+clean; 698 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:39 smithi040 ceph-mon[32945]: pgmap v573: 97 pgs: 97 active+clean; 698 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:37:41.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:41 smithi169 ceph-mon[33112]: pgmap v574: 97 pgs: 97 active+clean; 698 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:41 smithi040 ceph-mon[32945]: pgmap v574: 97 pgs: 97 active+clean; 698 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:41 smithi031 ceph-mon[29167]: pgmap v574: 97 pgs: 97 active+clean; 698 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:37:43.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:43 smithi169 ceph-mon[33112]: pgmap v575: 97 pgs: 97 active+clean; 698 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:37:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:43 smithi040 ceph-mon[32945]: pgmap v575: 97 pgs: 97 active+clean; 698 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:37:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:43 smithi031 ceph-mon[29167]: pgmap v575: 97 pgs: 97 active+clean; 698 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:37:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 -- 172.21.15.31:0/1385906257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108102300 msgr2=0x7f8108108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 --2- 172.21.15.31:0/1385906257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108102300 0x7f8108108b90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8100009ca0 tx=0x7f810002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 -- 172.21.15.31:0/1385906257 shutdown_connections 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 --2- 172.21.15.31:0/1385906257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108102300 0x7f8108108b90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 --2- 172.21.15.31:0/1385906257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8108101940 0x7f8108101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 --2- 172.21.15.31:0/1385906257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8108100750 0x7f8108100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.753+0000 7f81106a5640 1 -- 172.21.15.31:0/1385906257 >> 172.21.15.31:0/1385906257 conn(0x7f81080fbcc0 msgr2=0x7f81080fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 -- 172.21.15.31:0/1385906257 shutdown_connections 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 -- 172.21.15.31:0/1385906257 wait complete. 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 Processor -- start 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 -- start start 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8108100750 0x7f8108197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 0x7f8108197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8108102300 0x7f810819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f810810c210 con 0x7f8108101940 2024-04-25T00:37:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f810810c090 con 0x7f8108102300 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f81106a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f810810c390 con 0x7f8108100750 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f810dc40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 0x7f8108197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f810dc40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 0x7f8108197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39602/0 (socket says 172.21.15.31:39602) 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.754+0000 7f810dc40640 1 -- 172.21.15.31:0/2307623470 learned_addr learned my addr 172.21.15.31:0/2307623470 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810e441640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8108100750 0x7f8108197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810ec42640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8108102300 0x7f810819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810dc40640 1 -- 172.21.15.31:0/2307623470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8108100750 msgr2=0x7f8108197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810dc40640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8108100750 0x7f8108197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810dc40640 1 -- 172.21.15.31:0/2307623470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8108102300 msgr2=0x7f810819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810dc40640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8108102300 0x7f810819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810dc40640 1 -- 172.21.15.31:0/2307623470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810819f510 con 0x7f8108101940 2024-04-25T00:37:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f810dc40640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 0x7f8108197e80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f80f800d950 tx=0x7f80f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f80ff7fe640 1 -- 172.21.15.31:0/2307623470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8017070 con 0x7f8108101940 2024-04-25T00:37:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f80ff7fe640 1 -- 172.21.15.31:0/2307623470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f8009c30 con 0x7f8108101940 2024-04-25T00:37:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810819f800 con 0x7f8108101940 2024-04-25T00:37:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f80ff7fe640 1 -- 172.21.15.31:0/2307623470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f800f620 con 0x7f8108101940 2024-04-25T00:37:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.755+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810819fd50 con 0x7f8108101940 2024-04-25T00:37:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f80ff7fe640 1 -- 172.21.15.31:0/2307623470 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f80f80109a0 con 0x7f8108101940 2024-04-25T00:37:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 Processor -- start 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 -- start start 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739ce50 0x5574f739d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 0x5574f73a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5574f7300cb0 con 0x5574f739d840 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5574f72a95b0 con 0x5574f73a29b0 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.758+0000 7f81133b4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5574f72fcc70 con 0x5574f739ce50 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f80ff7fe640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f80f40a2c30 0x7f80f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 0x5574f73a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 0x5574f73a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39606/0 (socket says 172.21.15.31:39606) 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 -- 172.21.15.31:0/3608655290 learned_addr learned my addr 172.21.15.31:0/3608655290 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810ec42640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f80f40a2c30 0x7f80f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f80ff7fe640 1 -- 172.21.15.31:0/2307623470 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f80f80cecc0 con 0x7f8108101940 2024-04-25T00:37:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810dc40640 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810ec42640 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739ce50 0x5574f739d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 -- 172.21.15.31:0/3608655290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739ce50 msgr2=0x5574f739d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739ce50 0x5574f739d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 -- 172.21.15.31:0/3608655290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 msgr2=0x5574f73a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 -- 172.21.15.31:0/3608655290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f7015210 con 0x5574f739d840 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810dc40640 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810e441640 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 0x5574f73a2340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8104009ca0 tx=0x7f8104011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801f1941f2ec2065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f810ec42640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f80f40a2c30 0x7f80f40a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5574f739e0a0 tx=0x7f8100009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f80fcff9640 1 -- 172.21.15.31:0/3608655290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104004220 con 0x5574f739d840 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.759+0000 7f80fcff9640 1 -- 172.21.15.31:0/3608655290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81040043c0 con 0x5574f739d840 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f80fcff9640 1 -- 172.21.15.31:0/3608655290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104019760 con 0x5574f739d840 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 -- 172.21.15.31:0/3608655290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 msgr2=0x5574f73a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 0x5574f73a2340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8104009ca0 tx=0x7f8104011810 comp rx=0 tx=0).stop 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 -- 172.21.15.31:0/3608655290 shutdown_connections 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739d840 0x5574f73a2340 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 --2- 172.21.15.31:0/3608655290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739ce50 0x5574f739d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 -- 172.21.15.31:0/3608655290 >> 172.21.15.31:0/3608655290 conn(0x5574f73984c0 msgr2=0x5574f73988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 -- 172.21.15.31:0/3608655290 shutdown_connections 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.760+0000 7f81133b4740 1 -- 172.21.15.31:0/3608655290 wait complete. 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 -- start start 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 Processor -- start 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5574f73a5960 con 0x5574f739ce50 2024-04-25T00:37:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5574f72bcb50 con 0x5574f73a29b0 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5574f73a5ae0 con 0x5574f739d840 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f81133b4740 10 client.0 _ll_register_callbacks cb 0x5574f72f7ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73a7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39622/0 (socket says 172.21.15.31:39622) 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 -- 172.21.15.31:0/3773517976 learned_addr learned my addr 172.21.15.31:0/3773517976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810ec42640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810dc40640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 -- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 msgr2=0x5574f73a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 -- 172.21.15.31:0/3773517976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 msgr2=0x5574f73ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.761+0000 7f810e441640 1 -- 172.21.15.31:0/3773517976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f70e1620 con 0x5574f739ce50 2024-04-25T00:37:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f810e441640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73a7210 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8104009380 tx=0x7f81040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e37f3705dc15ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f80f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f80f1ffb640 1 -- 172.21.15.31:0/3773517976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104028070 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81040043c0 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f80f1ffb640 1 -- 172.21.15.31:0/3773517976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f8104005040 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f80f1ffb640 1 -- 172.21.15.31:0/3773517976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8104034050 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f81133b4740 10 client.15963 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574f73b1a70 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f80f1ffb640 1 -- 172.21.15.31:0/3773517976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81040043c0 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f80f1ffb640 1 -- 172.21.15.31:0/3773517976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f81040194a0 con 0x5574f739ce50 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f81133b4740 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f81133b4740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.762+0000 7f81133b4740 20 client.15963 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.763+0000 7f81133b4740 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f73b2fb0 0x5574f73b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.763+0000 7f81133b4740 4 client.15963 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.763+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574f730e1f0 con 0x5574f73b2fb0 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.763+0000 7f810dc40640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f73b2fb0 0x5574f73b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.763+0000 7f810dc40640 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f73b2fb0 0x5574f73b53a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dedd615734d15ee3 server_cookie=cb70de8a1084530f in_seq=0 out_seq=0 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.763+0000 7f80f1ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f80f1ffb640 1 -- 172.21.15.31:0/3773517976 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574f730e1f0 con 0x5574f73b2fb0 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f80f1ffb640 10 client.15963 handle_command_reply: tid=0 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 client.15963 shutdown 2024-04-25T00:37:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 msgr2=0x5574f73a7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73a7210 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8104009380 tx=0x7f81040038a0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 shutdown_connections 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f73b2fb0 0x5574f73b53a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 --2- 172.21.15.31:0/3773517976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73a7210 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f80f1ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 >> 172.21.15.31:0/3773517976 conn(0x5574f73984c0 msgr2=0x5574f7397440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 shutdown_connections 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.764+0000 7f81133b4740 1 -- 172.21.15.31:0/3773517976 wait complete. 2024-04-25T00:37:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.765+0000 7f81133b4740 20 client.15963 trim_cache size 0 max 16384 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.779+0000 7f81133b4740 1 Processor -- start 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.779+0000 7f81133b4740 1 -- start start 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f81133b4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f81133b4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f81133b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f81133b4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f81081a8ba0 con 0x5574f739ce50 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f81133b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81081aa6e0 con 0x5574f73a29b0 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f81133b4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f81081a9270 con 0x5574f739d840 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39636/0 (socket says 172.21.15.31:39636) 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 -- 172.21.15.31:0/1616134172 learned_addr learned my addr 172.21.15.31:0/1616134172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810dc40640 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 -- 172.21.15.31:0/1616134172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 msgr2=0x5574f73ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 -- 172.21.15.31:0/1616134172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 msgr2=0x5574f73a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 -- 172.21.15.31:0/1616134172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f73a1e50 con 0x5574f739ce50 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f810e441640 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73972c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8104019a80 tx=0x7f81040097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3f551d5cdd12ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f80f3fff640 1 -- 172.21.15.31:0/1616134172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810403d070 con 0x5574f739ce50 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f80f3fff640 1 -- 172.21.15.31:0/1616134172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81040042f0 con 0x5574f739ce50 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.780+0000 7f80f3fff640 1 -- 172.21.15.31:0/1616134172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104028040 con 0x5574f739ce50 2024-04-25T00:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 -- 172.21.15.31:0/1616134172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 msgr2=0x5574f73972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73972c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8104019a80 tx=0x7f81040097e0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 -- 172.21.15.31:0/1616134172 shutdown_connections 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f73a29b0 0x5574f73a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f739d840 0x5574f73ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 --2- 172.21.15.31:0/1616134172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73972c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 -- 172.21.15.31:0/1616134172 >> 172.21.15.31:0/1616134172 conn(0x5574f73984c0 msgr2=0x5574f7396bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 -- 172.21.15.31:0/1616134172 shutdown_connections 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 -- 172.21.15.31:0/1616134172 wait complete. 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 -- start start 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.781+0000 7f81133b4740 1 Processor -- start 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f739d840 0x5574f73ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f73a29b0 0x5574f7417510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f81081aa010 con 0x5574f739ce50 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81081a9940 con 0x5574f739d840 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f81081a6b80 con 0x5574f73a29b0 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 10 client.0 _ll_register_callbacks cb 0x7f8104009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39642/0 (socket says 172.21.15.31:39642) 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 -- 172.21.15.31:0/452756809 learned_addr learned my addr 172.21.15.31:0/452756809 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810ec42640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f73a29b0 0x5574f7417510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810dc40640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f739d840 0x5574f73ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 -- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f73a29b0 msgr2=0x5574f7417510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f73a29b0 0x5574f7417510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 -- 172.21.15.31:0/452756809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f739d840 msgr2=0x5574f73ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f739d840 0x5574f73ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 -- 172.21.15.31:0/452756809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f73ae970 con 0x5574f739ce50 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810dc40640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f739d840 0x5574f73ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f810e441640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73ab360 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8104019ad0 tx=0x7f8104005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51176fd86f388d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f80f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f80f1ffb640 1 -- 172.21.15.31:0/452756809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810403d070 con 0x5574f739ce50 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f80f1ffb640 1 -- 172.21.15.31:0/452756809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f8104005070 con 0x5574f739ce50 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.782+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81040042f0 con 0x5574f739ce50 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f80f1ffb640 1 -- 172.21.15.31:0/452756809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8104049050 con 0x5574f739ce50 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 10 client.15969 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574f7418a40 con 0x5574f739ce50 2024-04-25T00:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f80f1ffb640 1 -- 172.21.15.31:0/452756809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81040042f0 con 0x5574f739ce50 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f80f1ffb640 1 -- 172.21.15.31:0/452756809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8104004aa0 con 0x5574f739ce50 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 10 client.15969 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 20 client.15969 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f74193c0 0x5574f741b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 4 client.15969 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574f7308540 con 0x5574f74193c0 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.783+0000 7f810dc40640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f74193c0 0x5574f741b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.784+0000 7f80f1ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.784+0000 7f810dc40640 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f74193c0 0x5574f741b7e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1ce120f390b8d1 server_cookie=da2650332a20f9a5 in_seq=0 out_seq=0 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.799+0000 7f80f1ffb640 1 -- 172.21.15.31:0/452756809 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574f7308540 con 0x5574f74193c0 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.799+0000 7f80f1ffb640 10 client.15969 handle_command_reply: tid=0 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.799+0000 7f81133b4740 1 client.15969 shutdown 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 msgr2=0x5574f73ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73ab360 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8104019ad0 tx=0x7f8104005fa0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 shutdown_connections 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5574f74193c0 0x5574f741b7e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574f73a29b0 0x5574f7417510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f80f1ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5574f739d840 0x5574f73ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 --2- 172.21.15.31:0/452756809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f739ce50 0x5574f73ab360 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 >> 172.21.15.31:0/452756809 conn(0x5574f73984c0 msgr2=0x5574f739bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 shutdown_connections 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 1 -- 172.21.15.31:0/452756809 wait complete. 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81133b4740 20 client.15969 trim_cache size 0 max 16384 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:37:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:37:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:37:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.800+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f80f40a2c30 msgr2=0x7f80f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f80f40a2c30 0x7f80f40a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5574f739e0a0 tx=0x7f8100009360 comp rx=0 tx=0).stop 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 msgr2=0x7f8108197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 0x7f8108197e80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f80f800d950 tx=0x7f80f800de10 comp rx=0 tx=0).stop 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 shutdown_connections 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8108102300 0x7f810819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f80f40a2c30 0x7f80f40a50f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8108101940 0x7f8108197e80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 --2- 172.21.15.31:0/2307623470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8108100750 0x7f8108197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 >> 172.21.15.31:0/2307623470 conn(0x7f81080fbcc0 msgr2=0x7f81080fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 shutdown_connections 2024-04-25T00:37:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:43.801+0000 7f81106a5640 1 -- 172.21.15.31:0/2307623470 wait complete. 2024-04-25T00:37:45.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:45 smithi169 ceph-mon[33112]: pgmap v576: 97 pgs: 97 active+clean; 754 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:37:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:45 smithi040 ceph-mon[32945]: pgmap v576: 97 pgs: 97 active+clean; 754 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:37:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:45 smithi031 ceph-mon[29167]: pgmap v576: 97 pgs: 97 active+clean; 754 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:37:47.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:47 smithi169 ceph-mon[33112]: pgmap v577: 97 pgs: 97 active+clean; 754 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:37:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:47 smithi040 ceph-mon[32945]: pgmap v577: 97 pgs: 97 active+clean; 754 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:37:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:47 smithi031 ceph-mon[29167]: pgmap v577: 97 pgs: 97 active+clean; 754 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:37:49.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:49 smithi169 ceph-mon[33112]: pgmap v578: 97 pgs: 97 active+clean; 821 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T00:37:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:49 smithi040 ceph-mon[32945]: pgmap v578: 97 pgs: 97 active+clean; 821 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T00:37:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:49 smithi031 ceph-mon[29167]: pgmap v578: 97 pgs: 97 active+clean; 821 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T00:37:51.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:51 smithi169 ceph-mon[33112]: pgmap v579: 97 pgs: 97 active+clean; 821 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:37:51.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:51 smithi040 ceph-mon[32945]: pgmap v579: 97 pgs: 97 active+clean; 821 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:37:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:51 smithi031 ceph-mon[29167]: pgmap v579: 97 pgs: 97 active+clean; 821 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:37:52.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:52 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:37:52.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:52 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:37:52.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:52 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: pgmap v580: 97 pgs: 97 active+clean; 821 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:37:53.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:37:53.391 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: pgmap v580: 97 pgs: 97 active+clean; 821 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:37:53.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:37:53.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:37:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: pgmap v580: 97 pgs: 97 active+clean; 821 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:37:53.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:37:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:54 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-04-25T00:37:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:54 smithi169 ceph-mon[33112]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T00:37:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:54 smithi169 ceph-mon[33112]: Unable to set osd_memory_target on smithi040 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T00:37:54.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:54 smithi169 ceph-mon[33112]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T00:37:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:54 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-04-25T00:37:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:54 smithi040 ceph-mon[32945]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T00:37:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:54 smithi040 ceph-mon[32945]: Unable to set osd_memory_target on smithi040 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T00:37:54.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:54 smithi040 ceph-mon[32945]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T00:37:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:54 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-04-25T00:37:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:54 smithi031 ceph-mon[29167]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T00:37:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:54 smithi031 ceph-mon[29167]: Unable to set osd_memory_target on smithi040 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T00:37:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:54 smithi031 ceph-mon[29167]: Unable to set osd_memory_target on smithi169 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T00:37:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:55 smithi169 ceph-mon[33112]: pgmap v581: 97 pgs: 97 active+clean; 878 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:37:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:55 smithi040 ceph-mon[32945]: pgmap v581: 97 pgs: 97 active+clean; 878 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:37:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:55 smithi031 ceph-mon[29167]: pgmap v581: 97 pgs: 97 active+clean; 878 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 -- 172.21.15.31:0/1382094023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 msgr2=0x7fc2f4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 --2- 172.21.15.31:0/1382094023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4102ee0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009930 tx=0x7fc2dc02a420 comp rx=0 tx=0).stop 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 -- 172.21.15.31:0/1382094023 shutdown_connections 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 --2- 172.21.15.31:0/1382094023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f4103420 0x7fc2f4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 --2- 172.21.15.31:0/1382094023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4102ee0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 --2- 172.21.15.31:0/1382094023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f40ffa90 0x7fc2f40ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 -- 172.21.15.31:0/1382094023 >> 172.21.15.31:0/1382094023 conn(0x7fc2f40fb3c0 msgr2=0x7fc2f40fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 -- 172.21.15.31:0/1382094023 shutdown_connections 2024-04-25T00:37:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 -- 172.21.15.31:0/1382094023 wait complete. 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 Processor -- start 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.900+0000 7fc2fa361640 1 -- start start 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2fa361640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f40ffa90 0x7fc2f4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2fa361640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2fa361640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f4103420 0x7fc2f419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2fa361640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f410d320 con 0x7fc2f4102a60 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2fa361640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f410d1a0 con 0x7fc2f4103420 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2fa361640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f410d4a0 con 0x7fc2f40ffa90 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39676/0 (socket says 172.21.15.31:39676) 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3754253268 learned_addr learned my addr 172.21.15.31:0/3754253268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f40ffa90 0x7fc2f4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f4103420 0x7fc2f419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3754253268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f40ffa90 msgr2=0x7fc2f4197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f40ffa90 0x7fc2f4197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3754253268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f4103420 msgr2=0x7fc2f419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f4103420 0x7fc2f419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3754253268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f419f530 con 0x7fc2f4102a60 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.901+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4197e50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc004770 tx=0x7fc2dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.902+0000 7fc2f17fa640 1 -- 172.21.15.31:0/3754253268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc03f030 con 0x7fc2f4102a60 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.902+0000 7fc2f17fa640 1 -- 172.21.15.31:0/3754253268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2dc003aa0 con 0x7fc2f4102a60 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.902+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f419f7c0 con 0x7fc2f4102a60 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.902+0000 7fc2f17fa640 1 -- 172.21.15.31:0/3754253268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc035070 con 0x7fc2f4102a60 2024-04-25T00:37:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.902+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f419fcb0 con 0x7fc2f4102a60 2024-04-25T00:37:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.904+0000 7fc2f17fa640 1 -- 172.21.15.31:0/3754253268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc2dc042030 con 0x7fc2f4102a60 2024-04-25T00:37:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.904+0000 7fc2fd06f740 1 Processor -- start 2024-04-25T00:37:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 -- start start 2024-04-25T00:37:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a566680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566c50 0x55b66a56b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a56e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b66a4c5cb0 con 0x55b66a56bdc0 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b66a46e5b0 con 0x55b66a566c50 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b66a4c1c70 con 0x55b66a566260 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f17fa640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc2e00a2c30 0x7fc2e00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a56e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a56e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39686/0 (socket says 172.21.15.31:39686) 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3749284042 learned_addr learned my addr 172.21.15.31:0/3749284042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f88fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a566680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc2e00a2c30 0x7fc2e00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566c50 0x55b66a56b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f17fa640 1 -- 172.21.15.31:0/3754253268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc2dc0ef8b0 con 0x7fc2f4102a60 2024-04-25T00:37:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3749284042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 msgr2=0x55b66a566680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a566680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3749284042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566c50 msgr2=0x55b66a56b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566c50 0x55b66a56b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f37fe640 1 -- 172.21.15.31:0/3749284042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66a1da210 con 0x55b66a56bdc0 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.905+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a566680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a56e250 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0ed320 tx=0x7fc2dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732d2d69694ddb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc2e00a2c30 0x7fc2e00a50f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00afd0 tx=0x7fc2ec005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2ea7fc640 1 -- 172.21.15.31:0/3749284042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc049040 con 0x55b66a56bdc0 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2ea7fc640 1 -- 172.21.15.31:0/3749284042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2dc0bc080 con 0x55b66a56bdc0 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2ea7fc640 1 -- 172.21.15.31:0/3749284042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0dc070 con 0x55b66a56bdc0 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2fd06f740 1 -- 172.21.15.31:0/3749284042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 msgr2=0x55b66a56e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.906+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a56e250 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0ed320 tx=0x7fc2dc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- 172.21.15.31:0/3749284042 shutdown_connections 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a56e250 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566c50 0x55b66a56b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/3749284042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a566680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- 172.21.15.31:0/3749284042 >> 172.21.15.31:0/3749284042 conn(0x55b66a55d4c0 msgr2=0x55b66a55d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- 172.21.15.31:0/3749284042 shutdown_connections 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- 172.21.15.31:0/3749284042 wait complete. 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- start start 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 Processor -- start 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 0x55b66a570430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a570970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a5779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b66a56eb30 con 0x55b66a566260 2024-04-25T00:37:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b66a481b50 con 0x55b66a56bdc0 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b66a56ecb0 con 0x55b66a566c50 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.907+0000 7fc2fd06f740 10 client.0 _ll_register_callbacks cb 0x55b66a4bcca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 0x55b66a570430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 0x55b66a570430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39702/0 (socket says 172.21.15.31:39702) 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 -- 172.21.15.31:0/2768730327 learned_addr learned my addr 172.21.15.31:0/2768730327 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a5779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a570970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 -- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 msgr2=0x55b66a570970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a570970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 -- 172.21.15.31:0/2768730327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 msgr2=0x55b66a5779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a5779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 -- 172.21.15.31:0/2768730327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66a2a6620 con 0x55b66a566260 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a570970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 0x55b66a570430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40126e0 tx=0x7fc2e4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66a06d0dc266840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2768730327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4004050 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2768730327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc2e40041f0 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2dc0bc080 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2768730327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2e40043f0 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2fd06f740 10 client.15981 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.908+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b66a57adf0 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2768730327 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2dc0bc080 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2768730327 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc2e4025070 con 0x55b66a566260 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2fd06f740 10 client.15981 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2fd06f740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2fd06f740 20 client.15981 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a57c4e0 0x55b66a57e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2fd06f740 4 client.15981 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66a4d31f0 con 0x55b66a57c4e0 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.909+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a57c4e0 0x55b66a57e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.910+0000 7fc2d37fe640 10 client.15981 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.910+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a57c4e0 0x55b66a57e9a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0b38b5ef46e75f server_cookie=15993b6ba7b38761 in_seq=0 out_seq=0 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.910+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2768730327 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b66a4d31f0 con 0x55b66a57c4e0 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.910+0000 7fc2d37fe640 10 client.15981 handle_command_reply: tid=0 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 client.15981 shutdown 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 msgr2=0x55b66a570430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 0x55b66a570430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40126e0 tx=0x7fc2e4012ba0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 shutdown_connections 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a57c4e0 0x55b66a57e9a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a5779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a570970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2d37fe640 0 client.15981 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2768730327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566260 0x55b66a570430 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 >> 172.21.15.31:0/2768730327 conn(0x55b66a55d4c0 msgr2=0x55b66a55c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 shutdown_connections 2024-04-25T00:37:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2768730327 wait complete. 2024-04-25T00:37:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.911+0000 7fc2fd06f740 20 client.15981 trim_cache size 0 max 16384 2024-04-25T00:37:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 Processor -- start 2024-04-25T00:37:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 -- start start 2024-04-25T00:37:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a570710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 0x55b66a577970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a560950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a6920 con 0x55b66a566c50 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a8460 con 0x55b66a56bdc0 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a6ff0 con 0x55b66a566260 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 0x55b66a577970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 0x55b66a577970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39704/0 (socket says 172.21.15.31:39704) 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2f37fe640 1 -- 172.21.15.31:0/1057283864 learned_addr learned my addr 172.21.15.31:0/1057283864 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a560950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.926+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a570710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2f37fe640 1 -- 172.21.15.31:0/1057283864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 msgr2=0x55b66a570710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a570710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2f37fe640 1 -- 172.21.15.31:0/1057283864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 msgr2=0x55b66a560950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a560950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2f37fe640 1 -- 172.21.15.31:0/1057283864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66a55d060 con 0x55b66a566c50 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 0x55b66a577970 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0b8a50 tx=0x7fc2dc0a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63eae411149c4ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2e9ffb640 1 -- 172.21.15.31:0/1057283864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc049040 con 0x55b66a566c50 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2e9ffb640 1 -- 172.21.15.31:0/1057283864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2dc0c4cf0 con 0x55b66a566c50 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2e9ffb640 1 -- 172.21.15.31:0/1057283864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0dc0b0 con 0x55b66a566c50 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2fd06f740 1 -- 172.21.15.31:0/1057283864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 msgr2=0x55b66a577970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.927+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 0x55b66a577970 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0b8a50 tx=0x7fc2dc0a8040 comp rx=0 tx=0).stop 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- 172.21.15.31:0/1057283864 shutdown_connections 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a56bdc0 0x55b66a560950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a566c50 0x55b66a577970 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/1057283864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566260 0x55b66a570710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- 172.21.15.31:0/1057283864 >> 172.21.15.31:0/1057283864 conn(0x55b66a55d4c0 msgr2=0x55b66a560240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- 172.21.15.31:0/1057283864 shutdown_connections 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- 172.21.15.31:0/1057283864 wait complete. 2024-04-25T00:37:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- start start 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 Processor -- start 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566260 0x55b66a575320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a575860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a576800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a7d90 con 0x55b66a56bdc0 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a76c0 con 0x55b66a566260 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a4900 con 0x55b66a566c50 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.928+0000 7fc2fd06f740 10 client.0 _ll_register_callbacks cb 0x7fc2e4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a576800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a576800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39718/0 (socket says 172.21.15.31:39718) 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 -- 172.21.15.31:0/2476642363 learned_addr learned my addr 172.21.15.31:0/2476642363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566260 0x55b66a575320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f37fe640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a575860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 -- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 msgr2=0x55b66a575860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a575860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 -- 172.21.15.31:0/2476642363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566260 msgr2=0x55b66a575320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566260 0x55b66a575320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 -- 172.21.15.31:0/2476642363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66a576df0 con 0x55b66a56bdc0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2f88fe640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a576800 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00fcc0 tx=0x7fc2ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511db24e24d4ce59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2476642363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01dba0 con 0x55b66a56bdc0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2476642363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc2ec01dd40 con 0x55b66a56bdc0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2dc0c4cf0 con 0x55b66a56bdc0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2476642363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec004b20 con 0x55b66a56bdc0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2fd06f740 10 client.15987 fetch_fsmap learned FSMap version 26 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.929+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b66a5e2dd0 con 0x55b66a56bdc0 2024-04-25T00:37:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2476642363 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2dc0c4cf0 con 0x55b66a56bdc0 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2476642363 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc2ec0057d0 con 0x55b66a56bdc0 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2fd06f740 10 client.15987 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2fd06f740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2fd06f740 20 client.15987 populate_metadata read hostname 'smithi031' 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a5e3830 0x55b66a5e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2fd06f740 4 client.15987 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b66a4cd540 con 0x55b66a5e3830 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.930+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a5e3830 0x55b66a5e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.931+0000 7fc2d37fe640 10 client.15987 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.931+0000 7fc2f3fff640 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a5e3830 0x55b66a5e5c50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01f1b75940e766f server_cookie=6a6029fc754a0dd9 in_seq=0 out_seq=0 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2d37fe640 1 -- 172.21.15.31:0/2476642363 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b66a4cd540 con 0x55b66a5e3830 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2d37fe640 10 client.15987 handle_command_reply: tid=0 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 client.15987 shutdown 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 msgr2=0x55b66a576800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a576800 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00fcc0 tx=0x7fc2ec004ab0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 shutdown_connections 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55b66a5e3830 0x55b66a5e5c50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b66a56bdc0 0x55b66a576800 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b66a566c50 0x55b66a575860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 --2- 172.21.15.31:0/2476642363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b66a566260 0x55b66a575320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2d37fe640 0 client.15987 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 >> 172.21.15.31:0/2476642363 conn(0x55b66a55d4c0 msgr2=0x55b66a560ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 shutdown_connections 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 1 -- 172.21.15.31:0/2476642363 wait complete. 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.947+0000 7fc2fd06f740 20 client.15987 trim_cache size 0 max 16384 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:37:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:37:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc2e00a2c30 msgr2=0x7fc2e00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc2e00a2c30 0x7fc2e00a50f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00afd0 tx=0x7fc2ec005e90 comp rx=0 tx=0).stop 2024-04-25T00:37:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 msgr2=0x7fc2f4197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4197e50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc004770 tx=0x7fc2dc004270 comp rx=0 tx=0).stop 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 shutdown_connections 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f4103420 0x7fc2f419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc2e00a2c30 0x7fc2e00a50f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f4102a60 0x7fc2f4197e50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 --2- 172.21.15.31:0/3754253268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f40ffa90 0x7fc2f4197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 >> 172.21.15.31:0/3754253268 conn(0x7fc2f40fb3c0 msgr2=0x7fc2f4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 shutdown_connections 2024-04-25T00:37:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:37:56.948+0000 7fc2fa361640 1 -- 172.21.15.31:0/3754253268 wait complete. 2024-04-25T00:37:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:57 smithi169 ceph-mon[33112]: pgmap v582: 97 pgs: 97 active+clean; 878 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:57 smithi040 ceph-mon[32945]: pgmap v582: 97 pgs: 97 active+clean; 878 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:57 smithi031 ceph-mon[29167]: pgmap v582: 97 pgs: 97 active+clean; 878 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:37:59.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:37:59 smithi169 ceph-mon[33112]: pgmap v583: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:37:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:37:59 smithi040 ceph-mon[32945]: pgmap v583: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:37:59.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:37:59 smithi031 ceph-mon[29167]: pgmap v583: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:01.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:01 smithi169 ceph-mon[33112]: pgmap v584: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:38:01.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:01 smithi040 ceph-mon[32945]: pgmap v584: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:38:01.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:01 smithi031 ceph-mon[29167]: pgmap v584: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:38:03.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:03 smithi169 ceph-mon[33112]: pgmap v585: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:38:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:03 smithi040 ceph-mon[32945]: pgmap v585: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:38:03.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:03 smithi031 ceph-mon[29167]: pgmap v585: 97 pgs: 97 active+clean; 931 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:38:05.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:05 smithi169 ceph-mon[33112]: pgmap v586: 97 pgs: 97 active+clean; 999 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:05.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:05 smithi040 ceph-mon[32945]: pgmap v586: 97 pgs: 97 active+clean; 999 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:313/600: open fd = 3 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:314/600: open fd = 3 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:315/600: open fd = 3 2024-04-25T00:38:05.462 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:316/600: open fd = 3 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:317/600: open fd = 3 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:318/600: open fd = 3 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.463 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:319/600: open fd = 3 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:320/600: open fd = 3 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:321/600: open fd = 3 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:322/600: open fd = 3 2024-04-25T00:38:05.464 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:323/600: open fd = 3 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:324/600: open fd = 3 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:325/600: open fd = 3 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:326/600: open fd = 3 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:327/600: open fd = 3 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:328/600: open fd = 3 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:329/600: open fd = 3 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.466 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:330/600: open fd = 3 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:331/600: open fd = 3 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:332/600: open fd = 3 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.467 INFO:tasks.workunit.client.0.smithi031.stdout:333/600: open fd = 3 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:334/600: open fd = 3 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:335/600: open fd = 3 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:336/600: open fd = 3 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.468 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:337/600: open fd = 3 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:338/600: open fd = 3 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:339/600: open fd = 3 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:340/600: open fd = 3 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.469 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:341/600: open fd = 3 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:342/600: open fd = 3 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:343/600: open fd = 3 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.470 INFO:tasks.workunit.client.0.smithi031.stdout:344/600: open fd = 3 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:345/600: open fd = 3 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:346/600: open fd = 3 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:347/600: open fd = 3 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.471 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:348/600: open fd = 3 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:349/600: open fd = 3 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:350/600: open fd = 3 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:351/600: open fd = 3 2024-04-25T00:38:05.472 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.473 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.473 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.473 INFO:tasks.workunit.client.0.smithi031.stdout:352/600: open fd = 3 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:353/600: open fd = 3 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:354/600: open fd = 3 2024-04-25T00:38:05.474 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:355/600: open fd = 3 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:356/600: open fd = 3 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:357/600: open fd = 3 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:358/600: open fd = 3 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:359/600: open fd = 3 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:360/600: open fd = 3 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:361/600: open fd = 3 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.476 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:362/600: open fd = 3 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:363/600: open fd = 3 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:364/600: open fd = 3 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.477 INFO:tasks.workunit.client.0.smithi031.stdout:365/600: open fd = 3 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:366/600: open fd = 3 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:367/600: open fd = 3 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:368/600: open fd = 3 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:369/600: open fd = 3 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:370/600: open fd = 3 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:371/600: open fd = 3 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.479 INFO:tasks.workunit.client.0.smithi031.stdout:372/600: open fd = 3 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:373/600: open fd = 3 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:374/600: open fd = 3 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:38:05.480 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:38:05.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:05 smithi031 ceph-mon[29167]: pgmap v586: 97 pgs: 97 active+clean; 999 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:07.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:07 smithi169 ceph-mon[33112]: pgmap v587: 97 pgs: 97 active+clean; 999 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:07 smithi040 ceph-mon[32945]: pgmap v587: 97 pgs: 97 active+clean; 999 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:07 smithi031 ceph-mon[29167]: pgmap v587: 97 pgs: 97 active+clean; 999 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:09.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:09 smithi169 ceph-mon[33112]: pgmap v588: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:09 smithi040 ceph-mon[32945]: pgmap v588: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:09.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:09 smithi031 ceph-mon[29167]: pgmap v588: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 -- 172.21.15.31:0/3782091984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4241006a0 msgr2=0x7ff424100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 --2- 172.21.15.31:0/3782091984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4241006a0 0x7ff424100ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ca0 tx=0x7ff41402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 -- 172.21.15.31:0/3782091984 shutdown_connections 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 --2- 172.21.15.31:0/3782091984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff424102250 0x7ff424108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 --2- 172.21.15.31:0/3782091984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff424101890 0x7ff424101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 --2- 172.21.15.31:0/3782091984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4241006a0 0x7ff424100ac0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 -- 172.21.15.31:0/3782091984 >> 172.21.15.31:0/3782091984 conn(0x7ff4240fbc10 msgr2=0x7ff4240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 -- 172.21.15.31:0/3782091984 shutdown_connections 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 -- 172.21.15.31:0/3782091984 wait complete. 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.047+0000 7ff429631640 1 Processor -- start 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 -- start start 2024-04-25T00:38:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4241006a0 0x7ff424197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 0x7ff424197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff424102250 0x7ff42419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff42410d2f0 con 0x7ff424101890 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff42410d170 con 0x7ff4241006a0 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff429631640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff42410d470 con 0x7ff424102250 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 0x7ff424197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 0x7ff424197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58152/0 (socket says 172.21.15.31:58152) 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 -- 172.21.15.31:0/2355209483 learned_addr learned my addr 172.21.15.31:0/2355209483 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff422ffd640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4241006a0 0x7ff424197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 -- 172.21.15.31:0/2355209483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff424102250 msgr2=0x7ff42419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff424102250 0x7ff42419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 -- 172.21.15.31:0/2355209483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4241006a0 msgr2=0x7ff424197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4241006a0 0x7ff424197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.048+0000 7ff4227fc640 1 -- 172.21.15.31:0/2355209483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42419f5a0 con 0x7ff424101890 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.049+0000 7ff4227fc640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 0x7ff424197ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d770 tx=0x7ff40c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.049+0000 7ff41bfff640 1 -- 172.21.15.31:0/2355209483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c017070 con 0x7ff424101890 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.049+0000 7ff41bfff640 1 -- 172.21.15.31:0/2355209483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff40c0054d0 con 0x7ff424101890 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.049+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42419f890 con 0x7ff424101890 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.049+0000 7ff41bfff640 1 -- 172.21.15.31:0/2355209483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c004cf0 con 0x7ff424101890 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.049+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42419fd90 con 0x7ff424101890 2024-04-25T00:38:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.051+0000 7ff41bfff640 1 -- 172.21.15.31:0/2355209483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff40c004e90 con 0x7ff424101890 2024-04-25T00:38:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.051+0000 7ff42c41b740 1 Processor -- start 2024-04-25T00:38:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.051+0000 7ff42c41b740 1 -- start start 2024-04-25T00:38:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cc0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cc5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cc8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff42c41b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556ef4c1fcb0 con 0x556ef4cc0c50 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff42c41b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556ef4bc85b0 con 0x556ef4cc5dc0 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff42c41b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556ef4c1bc70 con 0x556ef4cc0260 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff41bfff640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4100a2c30 0x7ff4100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cc5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cc5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58154/0 (socket says 172.21.15.31:58154) 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 -- 172.21.15.31:0/1444939257 learned_addr learned my addr 172.21.15.31:0/1444939257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff4237fe640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4100a2c30 0x7ff4100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff4237fe640 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cc0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff41bfff640 1 -- 172.21.15.31:0/2355209483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff40c0ceb40 con 0x7ff424101890 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff4227fc640 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cc8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 -- 172.21.15.31:0/1444939257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 msgr2=0x556ef4cc0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cc0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 -- 172.21.15.31:0/1444939257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 msgr2=0x556ef4cc8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cc8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff422ffd640 1 -- 172.21.15.31:0/1444939257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef4934210 con 0x556ef4cc0c50 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.052+0000 7ff4227fc640 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cc8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff422ffd640 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cc5750 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff4241016f0 tx=0x7ff414009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fadcc5297cdf854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff4237fe640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4100a2c30 0x7ff4100a50f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x556ef4cc14b0 tx=0x7ff41c005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff4197fa640 1 -- 172.21.15.31:0/1444939257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4140316d0 con 0x556ef4cc0c50 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff4197fa640 1 -- 172.21.15.31:0/1444939257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff414036030 con 0x556ef4cc0c50 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff4197fa640 1 -- 172.21.15.31:0/1444939257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414004460 con 0x556ef4cc0c50 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff42c41b740 1 -- 172.21.15.31:0/1444939257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 msgr2=0x556ef4cc5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.053+0000 7ff42c41b740 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cc5750 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff4241016f0 tx=0x7ff414009a20 comp rx=0 tx=0).stop 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- 172.21.15.31:0/1444939257 shutdown_connections 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cc8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cc5750 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 --2- 172.21.15.31:0/1444939257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cc0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- 172.21.15.31:0/1444939257 >> 172.21.15.31:0/1444939257 conn(0x556ef4cb74c0 msgr2=0x556ef4cb78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- 172.21.15.31:0/1444939257 shutdown_connections 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- 172.21.15.31:0/1444939257 wait complete. 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- start start 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 Processor -- start 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 0x556ef4cca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc0c50 0x556ef4cca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556ef4cc8b90 con 0x556ef4cc0260 2024-04-25T00:38:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556ef4bdbb50 con 0x556ef4cc0c50 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556ef4cc8d10 con 0x556ef4cc5dc0 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff42c41b740 10 client.0 _ll_register_callbacks cb 0x556ef4c16ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff422ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 0x556ef4cca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff422ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 0x556ef4cca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58156/0 (socket says 172.21.15.31:58156) 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff422ffd640 1 -- 172.21.15.31:0/701209840 learned_addr learned my addr 172.21.15.31:0/701209840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.054+0000 7ff4227fc640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc0c50 0x556ef4cca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff4237fe640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff422ffd640 1 -- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc5dc0 msgr2=0x556ef4cd1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff422ffd640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff422ffd640 1 -- 172.21.15.31:0/701209840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc0c50 msgr2=0x556ef4cca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff422ffd640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc0c50 0x556ef4cca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff422ffd640 1 -- 172.21.15.31:0/701209840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef4a00620 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff422ffd640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 0x556ef4cca410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff414009430 tx=0x7ff414031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf66c6fe0b8ce4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff40a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff40a7fc640 1 -- 172.21.15.31:0/701209840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41404a040 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff40a7fc640 1 -- 172.21.15.31:0/701209840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff414003b70 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414036030 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff40a7fc640 1 -- 172.21.15.31:0/701209840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4140041c0 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff42c41b740 10 client.15999 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ef4cc8b90 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.055+0000 7ff40a7fc640 1 -- 172.21.15.31:0/701209840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff414036030 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff40a7fc640 1 -- 172.21.15.31:0/701209840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff41404d070 con 0x556ef4cc0260 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff42c41b740 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff42c41b740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff42c41b740 20 client.15999 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff42c41b740 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4cd60e0 0x556ef4cd84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff42c41b740 4 client.15999 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ef4c2d1f0 con 0x556ef4cd60e0 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff4227fc640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4cd60e0 0x556ef4cd84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff4227fc640 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4cd60e0 0x556ef4cd84d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc982df503a8f10 server_cookie=3d64b1c6c614facc in_seq=0 out_seq=0 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.056+0000 7ff40a7fc640 10 client.15999 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff40a7fc640 1 -- 172.21.15.31:0/701209840 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff40c080ae0 con 0x556ef4cd60e0 2024-04-25T00:38:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff40a7fc640 10 client.15999 handle_command_reply: tid=0 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 client.15999 shutdown 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 msgr2=0x556ef4cca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 0x556ef4cca410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff414009430 tx=0x7ff414031df0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 shutdown_connections 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4cd60e0 0x556ef4cd84d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc0c50 0x556ef4cca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 --2- 172.21.15.31:0/701209840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0260 0x556ef4cca410 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff40a7fc640 0 client.15999 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.057+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 >> 172.21.15.31:0/701209840 conn(0x556ef4cb74c0 msgr2=0x556ef4cb65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.058+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 shutdown_connections 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.058+0000 7ff42c41b740 1 -- 172.21.15.31:0/701209840 wait complete. 2024-04-25T00:38:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.058+0000 7ff42c41b740 20 client.15999 trim_cache size 0 max 16384 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 Processor -- start 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 -- start start 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cd1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cb6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a68e0 con 0x556ef4cc0c50 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a8420 con 0x556ef4cc5dc0 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff42c41b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a6fb0 con 0x556ef4cc0260 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff4227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cd1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff4227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cd1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58168/0 (socket says 172.21.15.31:58168) 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff4227fc640 1 -- 172.21.15.31:0/3367935236 learned_addr learned my addr 172.21.15.31:0/3367935236 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff422ffd640 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.073+0000 7ff4237fe640 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cb6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff4227fc640 1 -- 172.21.15.31:0/3367935236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 msgr2=0x556ef4cca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff4227fc640 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff4227fc640 1 -- 172.21.15.31:0/3367935236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 msgr2=0x556ef4cb6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff4227fc640 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cb6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff4227fc640 1 -- 172.21.15.31:0/3367935236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef4cc5270 con 0x556ef4cc0c50 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff4227fc640 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cd1950 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009610 tx=0x7ff40c0cc480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db092e088339191f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff418ff9640 1 -- 172.21.15.31:0/3367935236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0b3070 con 0x556ef4cc0c50 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff418ff9640 1 -- 172.21.15.31:0/3367935236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556ef4c2d1f0 con 0x556ef4cc0c50 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff418ff9640 1 -- 172.21.15.31:0/3367935236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c020040 con 0x556ef4cc0c50 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff42c41b740 1 -- 172.21.15.31:0/3367935236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 msgr2=0x556ef4cd1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.074+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cd1950 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009610 tx=0x7ff40c0cc480 comp rx=0 tx=0).stop 2024-04-25T00:38:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- 172.21.15.31:0/3367935236 shutdown_connections 2024-04-25T00:38:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cb6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4cd1950 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3367935236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4cca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- 172.21.15.31:0/3367935236 >> 172.21.15.31:0/3367935236 conn(0x556ef4cb74c0 msgr2=0x556ef4cb5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- 172.21.15.31:0/3367935236 shutdown_connections 2024-04-25T00:38:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- 172.21.15.31:0/3367935236 wait complete. 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- start start 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 Processor -- start 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4ccef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4ccf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a7d50 con 0x556ef4cc0c50 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a7680 con 0x556ef4cc5dc0 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a48c0 con 0x556ef4cc0260 2024-04-25T00:38:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff42c41b740 10 client.0 _ll_register_callbacks cb 0x7ff41402afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff4227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4ccf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff4227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4ccf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58180/0 (socket says 172.21.15.31:58180) 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff4227fc640 1 -- 172.21.15.31:0/3379563846 learned_addr learned my addr 172.21.15.31:0/3379563846 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.075+0000 7ff422ffd640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4ccef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4237fe640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4227fc640 1 -- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 msgr2=0x556ef4ccef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4227fc640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4ccef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4227fc640 1 -- 172.21.15.31:0/3379563846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 msgr2=0x556ef4cd0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4227fc640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4227fc640 1 -- 172.21.15.31:0/3379563846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef4cd0a50 con 0x556ef4cc0c50 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4237fe640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff4227fc640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4ccf450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0a3fc0 tx=0x7ff40c09b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57fc55558ed992c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff40a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff40a7fc640 1 -- 172.21.15.31:0/3379563846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c020070 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff40a7fc640 1 -- 172.21.15.31:0/3379563846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff40c0cc6c0 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ef4c2d1f0 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff40a7fc640 1 -- 172.21.15.31:0/3379563846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff40c0b3030 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff42c41b740 10 client.16005 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ef4d3c610 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff40a7fc640 1 -- 172.21.15.31:0/3379563846 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556ef4c2d1f0 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.076+0000 7ff40a7fc640 1 -- 172.21.15.31:0/3379563846 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff40c0a8080 con 0x556ef4cc0c50 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff42c41b740 10 client.16005 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff42c41b740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff42c41b740 20 client.16005 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4d3ce70 0x556ef4d3f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff42c41b740 4 client.16005 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ef4c27540 con 0x556ef4d3ce70 2024-04-25T00:38:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff422ffd640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4d3ce70 0x556ef4d3f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff422ffd640 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4d3ce70 0x556ef4d3f260 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93f9f2c1ada51ed server_cookie=3628b0ba7a334043 in_seq=0 out_seq=0 2024-04-25T00:38:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.077+0000 7ff40a7fc640 10 client.16005 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff40a7fc640 1 -- 172.21.15.31:0/3379563846 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ef4c27540 con 0x556ef4d3ce70 2024-04-25T00:38:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff40a7fc640 10 client.16005 handle_command_reply: tid=0 2024-04-25T00:38:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 client.16005 shutdown 2024-04-25T00:38:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 msgr2=0x556ef4ccf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4ccf450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0a3fc0 tx=0x7ff40c09b310 comp rx=0 tx=0).stop 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 shutdown_connections 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556ef4d3ce70 0x556ef4d3f260 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556ef4cc5dc0 0x556ef4cd0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef4cc0c50 0x556ef4ccf450 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff40a7fc640 0 client.16005 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 --2- 172.21.15.31:0/3379563846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef4cc0260 0x556ef4ccef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 >> 172.21.15.31:0/3379563846 conn(0x556ef4cb74c0 msgr2=0x556ef4cb65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 shutdown_connections 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 1 -- 172.21.15.31:0/3379563846 wait complete. 2024-04-25T00:38:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.093+0000 7ff42c41b740 20 client.16005 trim_cache size 0 max 16384 2024-04-25T00:38:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:38:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:38:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:38:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4100a2c30 msgr2=0x7ff4100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4100a2c30 0x7ff4100a50f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x556ef4cc14b0 tx=0x7ff41c005ef0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 msgr2=0x7ff424197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 0x7ff424197ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d770 tx=0x7ff40c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 shutdown_connections 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff424102250 0x7ff42419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff4100a2c30 0x7ff4100a50f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424101890 0x7ff424197ec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 --2- 172.21.15.31:0/2355209483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4241006a0 0x7ff424197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 >> 172.21.15.31:0/2355209483 conn(0x7ff4240fbc10 msgr2=0x7ff424104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 shutdown_connections 2024-04-25T00:38:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:10.094+0000 7ff429631640 1 -- 172.21.15.31:0/2355209483 wait complete. 2024-04-25T00:38:11.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:11 smithi169 ceph-mon[33112]: pgmap v589: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:38:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:11 smithi040 ceph-mon[32945]: pgmap v589: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:38:11.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:11 smithi031 ceph-mon[29167]: pgmap v589: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:38:13.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:13 smithi169 ceph-mon[33112]: pgmap v590: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:38:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:13 smithi040 ceph-mon[32945]: pgmap v590: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:38:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:13 smithi031 ceph-mon[29167]: pgmap v590: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:38:15.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:15 smithi169 ceph-mon[33112]: pgmap v591: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:38:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:15 smithi040 ceph-mon[32945]: pgmap v591: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:38:15.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:15 smithi031 ceph-mon[29167]: pgmap v591: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:38:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:17 smithi169 ceph-mon[33112]: pgmap v592: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:17.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:17 smithi040 ceph-mon[32945]: pgmap v592: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:17.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:17 smithi031 ceph-mon[29167]: pgmap v592: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:19.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:19 smithi169 ceph-mon[33112]: pgmap v593: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:19.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:19 smithi040 ceph-mon[32945]: pgmap v593: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:19.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:19 smithi031 ceph-mon[29167]: pgmap v593: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:21.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:21 smithi169 ceph-mon[33112]: pgmap v594: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:38:21.394 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:21 smithi040 ceph-mon[32945]: pgmap v594: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:38:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:21 smithi031 ceph-mon[29167]: pgmap v594: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 -- 172.21.15.31:0/2765750235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d81000a0 msgr2=0x7f66d81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 --2- 172.21.15.31:0/2765750235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d81000a0 0x7f66d81004c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009900 tx=0x7f66c802a410 comp rx=0 tx=0).stop 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 -- 172.21.15.31:0/2765750235 shutdown_connections 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 --2- 172.21.15.31:0/2765750235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81013a0 0x7f66d8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 --2- 172.21.15.31:0/2765750235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d8100a00 0x7f66d8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 --2- 172.21.15.31:0/2765750235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d81000a0 0x7f66d81004c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 -- 172.21.15.31:0/2765750235 >> 172.21.15.31:0/2765750235 conn(0x7f66d80fbc40 msgr2=0x7f66d80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 -- 172.21.15.31:0/2765750235 shutdown_connections 2024-04-25T00:38:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.194+0000 7f66ddc15640 1 -- 172.21.15.31:0/2765750235 wait complete. 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.195+0000 7f66ddc15640 1 Processor -- start 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.195+0000 7f66ddc15640 1 -- start start 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.195+0000 7f66ddc15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d81000a0 0x7f66d8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.195+0000 7f66ddc15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 0x7f66d819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.195+0000 7f66ddc15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81013a0 0x7f66d81a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.195+0000 7f66ddc15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810f4e0 con 0x7f66d8100a00 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66ddc15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810f360 con 0x7f66d81013a0 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66ddc15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810f660 con 0x7f66d81000a0 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 0x7f66d819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 0x7f66d819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45716/0 (socket says 172.21.15.31:45716) 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 -- 172.21.15.31:0/456486749 learned_addr learned my addr 172.21.15.31:0/456486749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d77fe640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d81000a0 0x7f66d8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d7fff640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81013a0 0x7f66d81a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 -- 172.21.15.31:0/456486749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d81000a0 msgr2=0x7f66d8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d81000a0 0x7f66d8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 -- 172.21.15.31:0/456486749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81013a0 msgr2=0x7f66d81a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81013a0 0x7f66d81a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 -- 172.21.15.31:0/456486749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d81a17a0 con 0x7f66d8100a00 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 0x7f66d819a0c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00d770 tx=0x7f66cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d4ff9640 1 -- 172.21.15.31:0/456486749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc017070 con 0x7f66d8100a00 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d4ff9640 1 -- 172.21.15.31:0/456486749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66cc0054d0 con 0x7f66d8100a00 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d81a1a90 con 0x7f66d8100a00 2024-04-25T00:38:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.196+0000 7f66d4ff9640 1 -- 172.21.15.31:0/456486749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc004cf0 con 0x7f66d8100a00 2024-04-25T00:38:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.197+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d81a1f90 con 0x7f66d8100a00 2024-04-25T00:38:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.199+0000 7f66d4ff9640 1 -- 172.21.15.31:0/456486749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f66cc004e90 con 0x7f66d8100a00 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 Processor -- start 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 -- start start 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262ae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f26211cb0 con 0x563f262ae840 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563f261ba5b0 con 0x563f262ade50 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66e0a01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563f2620dc70 con 0x563f262b39b0 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45724/0 (socket says 172.21.15.31:45724) 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 -- 172.21.15.31:0/3879417439 learned_addr learned my addr 172.21.15.31:0/3879417439 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d7fff640 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d4ff9640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66c00a2c30 0x7f66c00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 -- 172.21.15.31:0/3879417439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 msgr2=0x563f262b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d7fff640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66c00a2c30 0x7f66c00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 -- 172.21.15.31:0/3879417439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 msgr2=0x563f262ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d77fe640 1 -- 172.21.15.31:0/3879417439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f25f26210 con 0x563f262ae840 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.200+0000 7f66d4ff9640 1 -- 172.21.15.31:0/456486749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f66cc0cea90 con 0x7f66d8100a00 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66d77fe640 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262b3340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f66c802a930 tx=0x7f66c8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60347afe2a1d3259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66bdffb640 1 -- 172.21.15.31:0/3879417439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8035030 con 0x563f262ae840 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66d7fff640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66c00a2c30 0x7f66c00a50f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x563f262af0a0 tx=0x7f66d0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66bdffb640 1 -- 172.21.15.31:0/3879417439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66c8040070 con 0x563f262ae840 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66bdffb640 1 -- 172.21.15.31:0/3879417439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8003980 con 0x563f262ae840 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66e0a01740 1 -- 172.21.15.31:0/3879417439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 msgr2=0x563f262b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.201+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262b3340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f66c802a930 tx=0x7f66c8005e80 comp rx=0 tx=0).stop 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- 172.21.15.31:0/3879417439 shutdown_connections 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262b3340 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3879417439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- 172.21.15.31:0/3879417439 >> 172.21.15.31:0/3879417439 conn(0x563f262a94c0 msgr2=0x563f262a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- 172.21.15.31:0/3879417439 shutdown_connections 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- 172.21.15.31:0/3879417439 wait complete. 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- start start 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 Processor -- start 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f262b6900 con 0x563f262b39b0 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563f261cdb50 con 0x563f262ae840 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.202+0000 7f66e0a01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563f262b6a80 con 0x563f262ade50 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66e0a01740 10 client.0 _ll_register_callbacks cb 0x563f26208ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262bf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45730/0 (socket says 172.21.15.31:45730) 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d7fff640 1 -- 172.21.15.31:0/1316336865 learned_addr learned my addr 172.21.15.31:0/1316336865 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d77fe640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 -- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 msgr2=0x563f262b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 -- 172.21.15.31:0/1316336865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 msgr2=0x563f262bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262bf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 -- 172.21.15.31:0/1316336865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f25ff2620 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d7fff640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262b8780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66cc09b010 tx=0x7f66cc09b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed064a9cbc1e177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66b6ffd640 1 -- 172.21.15.31:0/1316336865 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc020050 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.203+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c8040070 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66b6ffd640 1 -- 172.21.15.31:0/1316336865 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f66cc0bb0b0 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66b6ffd640 1 -- 172.21.15.31:0/1316336865 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66cc009c30 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 10 client.25451 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f262c2b30 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66b6ffd640 1 -- 172.21.15.31:0/1316336865 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66c8040070 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66b6ffd640 1 -- 172.21.15.31:0/1316336865 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f66cc09f160 con 0x563f262ae840 2024-04-25T00:38:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 10 client.25451 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 20 client.25451 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f262c41c0 0x563f262c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 4 client.25451 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.204+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f2621f1f0 con 0x563f262c41c0 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.205+0000 7f66d77fe640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f262c41c0 0x563f262c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.205+0000 7f66d77fe640 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f262c41c0 0x563f262c6620 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657764c36d14db56 server_cookie=ce3084c4edfa93ef in_seq=0 out_seq=0 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.205+0000 7f66b6ffd640 10 client.25451 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66b6ffd640 1 -- 172.21.15.31:0/1316336865 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f2621f1f0 con 0x563f262c41c0 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66b6ffd640 10 client.25451 handle_command_reply: tid=0 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 client.25451 shutdown 2024-04-25T00:38:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 msgr2=0x563f262b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262b8780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66cc09b010 tx=0x7f66cc09b1d0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 shutdown_connections 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f262c41c0 0x563f262c6620 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262b8780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66b6ffd640 0 client.25451 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 --2- 172.21.15.31:0/1316336865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 >> 172.21.15.31:0/1316336865 conn(0x563f262a94c0 msgr2=0x563f262a85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 shutdown_connections 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 1 -- 172.21.15.31:0/1316336865 wait complete. 2024-04-25T00:38:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.206+0000 7f66e0a01740 20 client.25451 trim_cache size 0 max 16384 2024-04-25T00:38:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 Processor -- start 2024-04-25T00:38:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 -- start start 2024-04-25T00:38:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262ac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81aae30 con 0x563f262b39b0 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81ac970 con 0x563f262ae840 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.221+0000 7f66e0a01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81ab500 con 0x563f262ade50 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45740/0 (socket says 172.21.15.31:45740) 2024-04-25T00:38:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 -- 172.21.15.31:0/3139391364 learned_addr learned my addr 172.21.15.31:0/3139391364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 -- 172.21.15.31:0/3139391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 msgr2=0x563f262b3270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:38:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b3270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 -- 172.21.15.31:0/3139391364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 msgr2=0x563f262bf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262bf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 -- 172.21.15.31:0/3139391364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f262a9000 con 0x563f262b39b0 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66d7fff640 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262ac470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f66d0003be0 tx=0x7f66d0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea30ede2bc04406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66bd7fa640 1 -- 172.21.15.31:0/3139391364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004d20 con 0x563f262b39b0 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66bd7fa640 1 -- 172.21.15.31:0/3139391364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d00045c0 con 0x563f262b39b0 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.222+0000 7f66bd7fa640 1 -- 172.21.15.31:0/3139391364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0005840 con 0x563f262b39b0 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 -- 172.21.15.31:0/3139391364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 msgr2=0x563f262ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262ac470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f66d0003be0 tx=0x7f66d0005bd0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 -- 172.21.15.31:0/3139391364 shutdown_connections 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262b39b0 0x563f262ac470 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ae840 0x563f262bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 --2- 172.21.15.31:0/3139391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262ade50 0x563f262b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 -- 172.21.15.31:0/3139391364 >> 172.21.15.31:0/3139391364 conn(0x563f262a94c0 msgr2=0x563f262afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 -- 172.21.15.31:0/3139391364 shutdown_connections 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 -- 172.21.15.31:0/3139391364 wait complete. 2024-04-25T00:38:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 -- start start 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.223+0000 7f66e0a01740 1 Processor -- start 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81ac2a0 con 0x563f262ae840 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81abbd0 con 0x563f262ade50 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81a8e10 con 0x563f262b39b0 2024-04-25T00:38:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 10 client.0 _ll_register_callbacks cb 0x7f66cc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262bcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45750/0 (socket says 172.21.15.31:45750) 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 -- 172.21.15.31:0/189767305 learned_addr learned my addr 172.21.15.31:0/189767305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d77fe640 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 -- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 msgr2=0x563f262bdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 -- 172.21.15.31:0/189767305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 msgr2=0x563f262bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262bca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 -- 172.21.15.31:0/189767305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f262be540 con 0x563f262ae840 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66d6ffd640 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262bcf90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009ed0 tx=0x7f66cc00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb362002103656c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66b6ffd640 1 -- 172.21.15.31:0/189767305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0b3020 con 0x563f262ae840 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66b6ffd640 1 -- 172.21.15.31:0/189767305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f66cc09a760 con 0x563f262ae840 2024-04-25T00:38:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d00045c0 con 0x563f262ae840 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.224+0000 7f66b6ffd640 1 -- 172.21.15.31:0/189767305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66cc020040 con 0x563f262ae840 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 10 client.16023 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f262bf8c0 con 0x563f262ae840 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66b6ffd640 1 -- 172.21.15.31:0/189767305 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d00045c0 con 0x563f262ae840 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66b6ffd640 1 -- 172.21.15.31:0/189767305 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f66cc0a07a0 con 0x563f262ae840 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 10 client.16023 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 20 client.16023 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f2632abe0 0x563f2632d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 4 client.16023 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f26219540 con 0x563f2632abe0 2024-04-25T00:38:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.225+0000 7f66d77fe640 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f2632abe0 0x563f2632d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.226+0000 7f66d77fe640 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f2632abe0 0x563f2632d000 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2b6b99c8a058cb server_cookie=688a8a9c1cfb9892 in_seq=0 out_seq=0 2024-04-25T00:38:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.226+0000 7f66b6ffd640 10 client.16023 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66b6ffd640 1 -- 172.21.15.31:0/189767305 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f26219540 con 0x563f2632abe0 2024-04-25T00:38:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66b6ffd640 10 client.16023 handle_command_reply: tid=0 2024-04-25T00:38:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 client.16023 shutdown 2024-04-25T00:38:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 msgr2=0x563f262bcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262bcf90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009ed0 tx=0x7f66cc00df60 comp rx=0 tx=0).stop 2024-04-25T00:38:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 shutdown_connections 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563f2632abe0 0x563f2632d000 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563f262b39b0 0x563f262bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66b6ffd640 0 client.16023 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f262ae840 0x563f262bcf90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 --2- 172.21.15.31:0/189767305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f262ade50 0x563f262bca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 >> 172.21.15.31:0/189767305 conn(0x563f262a94c0 msgr2=0x563f262afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 shutdown_connections 2024-04-25T00:38:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 1 -- 172.21.15.31:0/189767305 wait complete. 2024-04-25T00:38:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.242+0000 7f66e0a01740 20 client.16023 trim_cache size 0 max 16384 2024-04-25T00:38:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:38:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:38:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:38:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66c00a2c30 msgr2=0x7f66c00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66c00a2c30 0x7f66c00a50f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x563f262af0a0 tx=0x7f66d0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 msgr2=0x7f66d819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 0x7f66d819a0c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00d770 tx=0x7f66cc00dc30 comp rx=0 tx=0).stop 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 shutdown_connections 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81013a0 0x7f66d81a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f66c00a2c30 0x7f66c00a50f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d8100a00 0x7f66d819a0c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 --2- 172.21.15.31:0/456486749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d81000a0 0x7f66d8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 >> 172.21.15.31:0/456486749 conn(0x7f66d80fbc40 msgr2=0x7f66d80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 shutdown_connections 2024-04-25T00:38:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:23.243+0000 7f66ddc15640 1 -- 172.21.15.31:0/456486749 wait complete. 2024-04-25T00:38:23.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:23 smithi169 ceph-mon[33112]: pgmap v595: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:23.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:23 smithi040 ceph-mon[32945]: pgmap v595: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:23.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:23 smithi031 ceph-mon[29167]: pgmap v595: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:25.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:25 smithi169 ceph-mon[33112]: pgmap v596: 97 pgs: 97 active+clean; 1.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:25.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:25 smithi040 ceph-mon[32945]: pgmap v596: 97 pgs: 97 active+clean; 1.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:25.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:25 smithi031 ceph-mon[29167]: pgmap v596: 97 pgs: 97 active+clean; 1.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:27.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:27 smithi040 ceph-mon[32945]: pgmap v597: 97 pgs: 97 active+clean; 1.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:38:27.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:27 smithi031 ceph-mon[29167]: pgmap v597: 97 pgs: 97 active+clean; 1.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:38:27.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:27 smithi169 ceph-mon[33112]: pgmap v597: 97 pgs: 97 active+clean; 1.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:38:29.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:29 smithi040 ceph-mon[32945]: pgmap v598: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T00:38:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:29 smithi031 ceph-mon[29167]: pgmap v598: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T00:38:29.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:29 smithi169 ceph-mon[33112]: pgmap v598: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T00:38:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:31 smithi031 ceph-mon[29167]: pgmap v599: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:38:31.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:31 smithi169 ceph-mon[33112]: pgmap v599: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:38:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:31 smithi040 ceph-mon[32945]: pgmap v599: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:38:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:33 smithi031 ceph-mon[29167]: pgmap v600: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:33.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:33 smithi169 ceph-mon[33112]: pgmap v600: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:33.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:33 smithi040 ceph-mon[32945]: pgmap v600: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:35 smithi031 ceph-mon[29167]: pgmap v601: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:38:35.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:35 smithi169 ceph-mon[33112]: pgmap v601: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:38:35.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:35 smithi040 ceph-mon[32945]: pgmap v601: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:38:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 -- 172.21.15.31:0/2115647942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be0102250 msgr2=0x7f8be0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2115647942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be0102250 0x7f8be0108ae0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009ca0 tx=0x7f8bd802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 -- 172.21.15.31:0/2115647942 shutdown_connections 2024-04-25T00:38:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2115647942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be0102250 0x7f8be0108ae0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2115647942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8be0101890 0x7f8be0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2115647942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be01006a0 0x7f8be0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 -- 172.21.15.31:0/2115647942 >> 172.21.15.31:0/2115647942 conn(0x7f8be00fbc10 msgr2=0x7f8be00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 -- 172.21.15.31:0/2115647942 shutdown_connections 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.344+0000 7f8be7cee640 1 -- 172.21.15.31:0/2115647942 wait complete. 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 Processor -- start 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 -- start start 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 0x7f8be0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be0101890 0x7f8be019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8be0102250 0x7f8be01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010d2f0 con 0x7f8be01006a0 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010d170 con 0x7f8be0101890 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be7cee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010d470 con 0x7f8be0102250 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 0x7f8be0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 0x7f8be0199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52646/0 (socket says 172.21.15.31:52646) 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 -- 172.21.15.31:0/2575858959 learned_addr learned my addr 172.21.15.31:0/2575858959 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5289640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be0101890 0x7f8be019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 -- 172.21.15.31:0/2575858959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8be0102250 msgr2=0x7f8be01a1080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8be0102250 0x7f8be01a1080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 -- 172.21.15.31:0/2575858959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be0101890 msgr2=0x7f8be019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be0101890 0x7f8be019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5a8a640 1 -- 172.21.15.31:0/2575858959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01a16f0 con 0x7f8be01006a0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.345+0000 7f8be5289640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be0101890 0x7f8be019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.346+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 0x7f8be0199ad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9c0 tx=0x7f8bdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.346+0000 7f8bd6ffd640 1 -- 172.21.15.31:0/2575858959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00cd80 con 0x7f8be01006a0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.346+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01a19e0 con 0x7f8be01006a0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.346+0000 7f8bd6ffd640 1 -- 172.21.15.31:0/2575858959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bdc0035f0 con 0x7f8be01006a0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.346+0000 7f8bd6ffd640 1 -- 172.21.15.31:0/2575858959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc010690 con 0x7f8be01006a0 2024-04-25T00:38:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.346+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01a1f30 con 0x7f8be01006a0 2024-04-25T00:38:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.348+0000 7f8bd6ffd640 1 -- 172.21.15.31:0/2575858959 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8bdc0037a0 con 0x7f8be01006a0 2024-04-25T00:38:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 Processor -- start 2024-04-25T00:38:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 -- start start 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb799270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb799840 0x5559bb79e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb79e9b0 0x5559bb7a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5559bb6fccb0 con 0x5559bb798e50 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5559bb6a55b0 con 0x5559bb799840 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5559bb6f8c70 con 0x5559bb79e9b0 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bd6ffd640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bcc0a2c30 0x7f8bcc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8be628b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb799270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8be628b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb799270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52660/0 (socket says 172.21.15.31:52660) 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8be628b640 1 -- 172.21.15.31:0/3662030676 learned_addr learned my addr 172.21.15.31:0/3662030676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8be628b640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bcc0a2c30 0x7f8bcc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8bd6ffd640 1 -- 172.21.15.31:0/2575858959 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f8bdc0cf590 con 0x7f8be01006a0 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb799840 0x5559bb79e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.349+0000 7f8be5289640 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb79e9b0 0x5559bb7a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 -- 172.21.15.31:0/3662030676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb79e9b0 msgr2=0x5559bb7a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb79e9b0 0x5559bb7a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 -- 172.21.15.31:0/3662030676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb799840 msgr2=0x5559bb79e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb799840 0x5559bb79e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 -- 172.21.15.31:0/3662030676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bb411210 con 0x5559bb798e50 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb799840 0x5559bb79e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb799270 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80097b0 tx=0x7f8bd8032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb5a06cbd9a92cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8be628b640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bcc0a2c30 0x7f8bcc0a50f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bd802ade0 tx=0x7f8bd8047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8bcbfff640 1 -- 172.21.15.31:0/3662030676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8003970 con 0x5559bb798e50 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8bcbfff640 1 -- 172.21.15.31:0/3662030676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd804e070 con 0x5559bb798e50 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.350+0000 7f8bcbfff640 1 -- 172.21.15.31:0/3662030676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8004700 con 0x5559bb798e50 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3662030676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 msgr2=0x5559bb799270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb799270 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80097b0 tx=0x7f8bd8032e70 comp rx=0 tx=0).stop 2024-04-25T00:38:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3662030676 shutdown_connections 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb79e9b0 0x5559bb7a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb799840 0x5559bb79e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3662030676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb799270 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3662030676 >> 172.21.15.31:0/3662030676 conn(0x5559bb7944c0 msgr2=0x5559bb7948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3662030676 shutdown_connections 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3662030676 wait complete. 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.351+0000 7f8bea9f9740 1 -- start start 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 Processor -- start 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5559bb7a1900 con 0x5559bb799840 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5559bb6b8b50 con 0x5559bb79e9b0 2024-04-25T00:38:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5559bb7a1a80 con 0x5559bb798e50 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8bea9f9740 10 client.0 _ll_register_callbacks cb 0x5559bb6f3ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8be5289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8be5289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52666/0 (socket says 172.21.15.31:52666) 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8be5289640 1 -- 172.21.15.31:0/3206183429 learned_addr learned my addr 172.21.15.31:0/3206183429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.352+0000 7f8be628b640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be5289640 1 -- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 msgr2=0x5559bb7a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be5289640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be5289640 1 -- 172.21.15.31:0/3206183429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 msgr2=0x5559bb7aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be5289640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7aa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be5289640 1 -- 172.21.15.31:0/3206183429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bb4dd620 con 0x5559bb799840 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be628b640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8be5289640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a37a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0013790 tx=0x7f8bd0013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347a1a7026d3ad80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bc97fa640 1 -- 172.21.15.31:0/3206183429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0020060 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bc97fa640 1 -- 172.21.15.31:0/3206183429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f8bd0011d70 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd804e070 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bc97fa640 1 -- 172.21.15.31:0/3206183429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bd000f450 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bea9f9740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559bb7adc20 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.353+0000 7f8bc97fa640 1 -- 172.21.15.31:0/3206183429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd804e070 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bc97fa640 1 -- 172.21.15.31:0/3206183429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bd001a030 con 0x5559bb799840 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bea9f9740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bea9f9740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bea9f9740 20 client.16035 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb7af3b0 0x5559bb7b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bea9f9740 4 client.16035 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559bb70a1f0 con 0x5559bb7af3b0 2024-04-25T00:38:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb7af3b0 0x5559bb7b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8bc97fa640 10 client.16035 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.354+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb7af3b0 0x5559bb7b1810 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9feda146d46f786 server_cookie=88dd698add04fb1d in_seq=0 out_seq=0 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.355+0000 7f8bc97fa640 1 -- 172.21.15.31:0/3206183429 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559bb70a1f0 con 0x5559bb7af3b0 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.355+0000 7f8bc97fa640 10 client.16035 handle_command_reply: tid=0 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.355+0000 7f8bea9f9740 1 client.16035 shutdown 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.355+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 msgr2=0x5559bb7a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.355+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a37a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0013790 tx=0x7f8bd0013c50 comp rx=0 tx=0).stop 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.355+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 shutdown_connections 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb7af3b0 0x5559bb7b1810 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a37a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3206183429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bc97fa640 0 client.16035 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 >> 172.21.15.31:0/3206183429 conn(0x5559bb7944c0 msgr2=0x5559bb7935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 shutdown_connections 2024-04-25T00:38:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3206183429 wait complete. 2024-04-25T00:38:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.356+0000 7f8bea9f9740 20 client.16035 trim_cache size 0 max 16384 2024-04-25T00:38:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 Processor -- start 2024-04-25T00:38:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 -- start start 2024-04-25T00:38:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb79ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb799840 0x5559bb7aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb79b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01aade0 con 0x5559bb798e50 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01ac920 con 0x5559bb79e9b0 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01ab4b0 con 0x5559bb799840 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb79ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb79ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52670/0 (socket says 172.21.15.31:52670) 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 -- 172.21.15.31:0/3957528683 learned_addr learned my addr 172.21.15.31:0/3957528683 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be628b640 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb79b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5289640 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb799840 0x5559bb7aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 -- 172.21.15.31:0/3957528683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb799840 msgr2=0x5559bb7aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb799840 0x5559bb7aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 -- 172.21.15.31:0/3957528683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 msgr2=0x5559bb79b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb79b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5a8a640 1 -- 172.21.15.31:0/3957528683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bb79df90 con 0x5559bb798e50 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.371+0000 7f8be5289640 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb799840 0x5559bb7aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb79ad10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0cd3d0 tx=0x7f8bdc0980a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5418460cbeeee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bcb7fe640 1 -- 172.21.15.31:0/3957528683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc09bdc0 con 0x5559bb798e50 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bcb7fe640 1 -- 172.21.15.31:0/3957528683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bdc0a0170 con 0x5559bb798e50 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bcb7fe640 1 -- 172.21.15.31:0/3957528683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc016330 con 0x5559bb798e50 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3957528683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 msgr2=0x5559bb79ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb79ad10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0cd3d0 tx=0x7f8bdc0980a0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3957528683 shutdown_connections 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb79b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb799840 0x5559bb7aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/3957528683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb798e50 0x5559bb79ad10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3957528683 >> 172.21.15.31:0/3957528683 conn(0x5559bb7944c0 msgr2=0x5559bb7a3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3957528683 shutdown_connections 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.372+0000 7f8bea9f9740 1 -- 172.21.15.31:0/3957528683 wait complete. 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 -- start start 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 Processor -- start 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01ac250 con 0x5559bb799840 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01abb80 con 0x5559bb79e9b0 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01a8dc0 con 0x5559bb798e50 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8bea9f9740 10 client.0 _ll_register_callbacks cb 0x7f8bd00096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8be5289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8be5289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52676/0 (socket says 172.21.15.31:52676) 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8be5289640 1 -- 172.21.15.31:0/2845551699 learned_addr learned my addr 172.21.15.31:0/2845551699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8be628b640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.373+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8be5289640 1 -- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 msgr2=0x5559bb7a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8be5289640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8be5289640 1 -- 172.21.15.31:0/2845551699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 msgr2=0x5559bb7a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8be5289640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8be5289640 1 -- 172.21.15.31:0/2845551699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bb7a9c90 con 0x5559bb799840 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8be5289640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a8690 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00117b0 tx=0x7f8bd0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800c6777dc8a5dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bc97fa640 1 -- 172.21.15.31:0/2845551699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011d70 con 0x5559bb799840 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bc97fa640 1 -- 172.21.15.31:0/2845551699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f8bd0005020 con 0x5559bb799840 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bdc0a0170 con 0x5559bb799840 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bc97fa640 1 -- 172.21.15.31:0/2845551699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bd0020030 con 0x5559bb799840 2024-04-25T00:38:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bea9f9740 10 client.16041 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559bb815b70 con 0x5559bb799840 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.374+0000 7f8bc97fa640 1 -- 172.21.15.31:0/2845551699 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bdc0a0170 con 0x5559bb799840 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bc97fa640 1 -- 172.21.15.31:0/2845551699 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bd001a040 con 0x5559bb799840 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bea9f9740 10 client.16041 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bea9f9740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bea9f9740 20 client.16041 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb8165c0 0x5559bb8189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bea9f9740 4 client.16041 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559bb704540 con 0x5559bb8165c0 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb8165c0 0x5559bb8189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8bc97fa640 10 client.16041 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.375+0000 7f8be5a8a640 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb8165c0 0x5559bb8189e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41bae49ba340bec server_cookie=b896ed8c17321900 in_seq=0 out_seq=0 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bc97fa640 1 -- 172.21.15.31:0/2845551699 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559bb704540 con 0x5559bb8165c0 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bc97fa640 10 client.16041 handle_command_reply: tid=0 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 client.16041 shutdown 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 msgr2=0x5559bb7a8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a8690 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00117b0 tx=0x7f8bd0009870 comp rx=0 tx=0).stop 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 shutdown_connections 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5559bb8165c0 0x5559bb8189e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559bb79e9b0 0x5559bb7a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559bb799840 0x5559bb7a8690 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 --2- 172.21.15.31:0/2845551699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5559bb798e50 0x5559bb7a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bc97fa640 0 client.16041 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 >> 172.21.15.31:0/2845551699 conn(0x5559bb7944c0 msgr2=0x5559bb7935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 shutdown_connections 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 1 -- 172.21.15.31:0/2845551699 wait complete. 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.392+0000 7f8bea9f9740 20 client.16041 trim_cache size 0 max 16384 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:38:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:38:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bcc0a2c30 msgr2=0x7f8bcc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bcc0a2c30 0x7f8bcc0a50f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bd802ade0 tx=0x7f8bd8047000 comp rx=0 tx=0).stop 2024-04-25T00:38:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 msgr2=0x7f8be0199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 0x7f8be0199ad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9c0 tx=0x7f8bdc00ee80 comp rx=0 tx=0).stop 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 shutdown_connections 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8be0102250 0x7f8be01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be0101890 0x7f8be019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8bcc0a2c30 0x7f8bcc0a50f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 --2- 172.21.15.31:0/2575858959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be01006a0 0x7f8be0199ad0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 >> 172.21.15.31:0/2575858959 conn(0x7f8be00fbc10 msgr2=0x7f8be01094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 shutdown_connections 2024-04-25T00:38:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:36.393+0000 7f8be7cee640 1 -- 172.21.15.31:0/2575858959 wait complete. 2024-04-25T00:38:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:37 smithi031 ceph-mon[29167]: pgmap v602: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:37.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:37 smithi169 ceph-mon[33112]: pgmap v602: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:37.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:37 smithi040 ceph-mon[32945]: pgmap v602: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:39.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:39 smithi031 ceph-mon[29167]: pgmap v603: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-25T00:38:39.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:39 smithi169 ceph-mon[33112]: pgmap v603: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-25T00:38:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:39 smithi040 ceph-mon[32945]: pgmap v603: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-25T00:38:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:41 smithi031 ceph-mon[29167]: pgmap v604: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 20 op/s 2024-04-25T00:38:41.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:41 smithi169 ceph-mon[33112]: pgmap v604: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 20 op/s 2024-04-25T00:38:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:41 smithi040 ceph-mon[32945]: pgmap v604: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 20 op/s 2024-04-25T00:38:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:43 smithi031 ceph-mon[29167]: pgmap v605: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:43.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:43 smithi169 ceph-mon[33112]: pgmap v605: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:43 smithi040 ceph-mon[32945]: pgmap v605: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:45 smithi031 ceph-mon[29167]: pgmap v606: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:45.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:45 smithi169 ceph-mon[33112]: pgmap v606: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:45.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:45 smithi040 ceph-mon[32945]: pgmap v606: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:47 smithi031 ceph-mon[29167]: pgmap v607: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:47.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:47 smithi169 ceph-mon[33112]: pgmap v607: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:47.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:47 smithi040 ceph-mon[32945]: pgmap v607: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:38:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 -- 172.21.15.31:0/3994370591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c102300 msgr2=0x7fba9c108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/3994370591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c102300 0x7fba9c108b90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ca0 tx=0x7fba9402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 -- 172.21.15.31:0/3994370591 shutdown_connections 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/3994370591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c102300 0x7fba9c108b90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/3994370591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c101940 0x7fba9c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/3994370591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba9c100750 0x7fba9c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 -- 172.21.15.31:0/3994370591 >> 172.21.15.31:0/3994370591 conn(0x7fba9c0fbcc0 msgr2=0x7fba9c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 -- 172.21.15.31:0/3994370591 shutdown_connections 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.492+0000 7fbaa25a5640 1 -- 172.21.15.31:0/3994370591 wait complete. 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.493+0000 7fbaa25a5640 1 Processor -- start 2024-04-25T00:38:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.493+0000 7fbaa25a5640 1 -- start start 2024-04-25T00:38:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.493+0000 7fbaa25a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c100750 0x7fba9c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.493+0000 7fbaa25a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 0x7fba9c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.493+0000 7fbaa25a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba9c102300 0x7fba9c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.493+0000 7fbaa25a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c10d330 con 0x7fba9c101940 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fbaa25a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c10d1b0 con 0x7fba9c100750 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fbaa25a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c10d4b0 con 0x7fba9c102300 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 0x7fba9c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 0x7fba9c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38782/0 (socket says 172.21.15.31:38782) 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 -- 172.21.15.31:0/2816760720 learned_addr learned my addr 172.21.15.31:0/2816760720 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9bfff640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c100750 0x7fba9c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba9c102300 0x7fba9c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 -- 172.21.15.31:0/2816760720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba9c102300 msgr2=0x7fba9c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba9c102300 0x7fba9c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 -- 172.21.15.31:0/2816760720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c100750 msgr2=0x7fba9c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c100750 0x7fba9c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 -- 172.21.15.31:0/2816760720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c1a16b0 con 0x7fba9c101940 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 0x7fba9c199fd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d950 tx=0x7fba8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba997fa640 1 -- 172.21.15.31:0/2816760720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84017070 con 0x7fba9c101940 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba997fa640 1 -- 172.21.15.31:0/2816760720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba84009c30 con 0x7fba9c101940 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fba997fa640 1 -- 172.21.15.31:0/2816760720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8400f620 con 0x7fba9c101940 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.494+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c1a19a0 con 0x7fba9c101940 2024-04-25T00:38:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.495+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c1a1ef0 con 0x7fba9c101940 2024-04-25T00:38:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.497+0000 7fba997fa640 1 -- 172.21.15.31:0/2816760720 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fba840109a0 con 0x7fba9c101940 2024-04-25T00:38:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.497+0000 7fbaa538e740 1 Processor -- start 2024-04-25T00:38:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.497+0000 7fbaa538e740 1 -- start start 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea0e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0e840 0x56502ea13340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea15e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa538e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56502e971cb0 con 0x56502ea139b0 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa538e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56502e91a5b0 con 0x56502ea0de50 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa538e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56502e96dc70 con 0x56502ea0e840 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba997fa640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fba880a2c30 0x7fba880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea15e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea15e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38796/0 (socket says 172.21.15.31:38796) 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9b7fe640 1 -- 172.21.15.31:0/814197276 learned_addr learned my addr 172.21.15.31:0/814197276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9bfff640 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0e840 0x56502ea13340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fba880a2c30 0x7fba880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea0e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba997fa640 1 -- 172.21.15.31:0/2816760720 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fba840cec70 con 0x7fba9c101940 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9bfff640 1 -- 172.21.15.31:0/814197276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 msgr2=0x56502ea0e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9bfff640 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea0e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9bfff640 1 -- 172.21.15.31:0/814197276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 msgr2=0x56502ea15e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9bfff640 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea15e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9bfff640 1 -- 172.21.15.31:0/814197276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502e686210 con 0x56502ea0e840 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea15e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.498+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fba880a2c30 0x7fba880a50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56502ea0f0a0 tx=0x7fba94005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fba9bfff640 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0e840 0x56502ea13340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00bbc0 tx=0x7fba8c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5f028307c33460 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fba927fc640 1 -- 172.21.15.31:0/814197276 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c004030 con 0x56502ea0e840 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fba927fc640 1 -- 172.21.15.31:0/814197276 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba8c005020 con 0x56502ea0e840 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fba927fc640 1 -- 172.21.15.31:0/814197276 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c004be0 con 0x56502ea0e840 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 -- 172.21.15.31:0/814197276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0e840 msgr2=0x56502ea13340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0e840 0x56502ea13340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00bbc0 tx=0x7fba8c0107a0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 -- 172.21.15.31:0/814197276 shutdown_connections 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea15e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0e840 0x56502ea13340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 --2- 172.21.15.31:0/814197276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea0e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.499+0000 7fbaa538e740 1 -- 172.21.15.31:0/814197276 >> 172.21.15.31:0/814197276 conn(0x56502ea094c0 msgr2=0x56502ea098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 -- 172.21.15.31:0/814197276 shutdown_connections 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 -- 172.21.15.31:0/814197276 wait complete. 2024-04-25T00:38:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 -- start start 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 Processor -- start 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea18280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0e840 0x56502ea187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56502ea16960 con 0x56502ea139b0 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56502e92db50 con 0x56502ea0e840 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56502ea16ae0 con 0x56502ea0de50 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.500+0000 7fbaa538e740 10 client.0 _ll_register_callbacks cb 0x56502e968ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea1f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38802/0 (socket says 172.21.15.31:38802) 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 -- 172.21.15.31:0/658618370 learned_addr learned my addr 172.21.15.31:0/658618370 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba9bfff640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea18280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0e840 0x56502ea187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 -- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 msgr2=0x56502ea18280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea18280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 -- 172.21.15.31:0/658618370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0e840 msgr2=0x56502ea187c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0e840 0x56502ea187c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 -- 172.21.15.31:0/658618370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502e752620 con 0x56502ea139b0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0e840 0x56502ea187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea1f800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fba940318d0 tx=0x7fba940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1a9556ce1b10ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba7b7fe640 1 -- 172.21.15.31:0/658618370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94052070 con 0x56502ea139b0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba7b7fe640 1 -- 172.21.15.31:0/658618370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fba94004250 con 0x56502ea139b0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba8c005020 con 0x56502ea139b0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fba7b7fe640 1 -- 172.21.15.31:0/658618370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba9405e050 con 0x56502ea139b0 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa538e740 10 client.16053 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.501+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56502ea22b50 con 0x56502ea139b0 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fba7b7fe640 1 -- 172.21.15.31:0/658618370 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba8c005020 con 0x56502ea139b0 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fba7b7fe640 1 -- 172.21.15.31:0/658618370 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba9404c450 con 0x56502ea139b0 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fbaa538e740 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fbaa538e740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fbaa538e740 20 client.16053 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fbaa538e740 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea24240 0x56502ea266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fbaa538e740 4 client.16053 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56502e97f1f0 con 0x56502ea24240 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.502+0000 7fba9bfff640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea24240 0x56502ea266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.503+0000 7fba9bfff640 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea24240 0x56502ea266a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99a52317eadd1f8 server_cookie=fcf632409ae59f67 in_seq=0 out_seq=0 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.503+0000 7fba7b7fe640 10 client.16053 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.503+0000 7fba7b7fe640 1 -- 172.21.15.31:0/658618370 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56502e97f1f0 con 0x56502ea24240 2024-04-25T00:38:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.503+0000 7fba7b7fe640 10 client.16053 handle_command_reply: tid=0 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.503+0000 7fbaa538e740 1 client.16053 shutdown 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 msgr2=0x56502ea1f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea1f800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fba940318d0 tx=0x7fba940095c0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 shutdown_connections 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea24240 0x56502ea266a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea139b0 0x56502ea1f800 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0e840 0x56502ea187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 --2- 172.21.15.31:0/658618370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea18280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fba7b7fe640 0 client.16053 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 >> 172.21.15.31:0/658618370 conn(0x56502ea094c0 msgr2=0x56502ea085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 shutdown_connections 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 1 -- 172.21.15.31:0/658618370 wait complete. 2024-04-25T00:38:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.504+0000 7fbaa538e740 20 client.16053 trim_cache size 0 max 16384 2024-04-25T00:38:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 Processor -- start 2024-04-25T00:38:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 -- start start 2024-04-25T00:38:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea139b0 0x56502ea2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1aadd0 con 0x56502ea0e840 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1ac910 con 0x56502ea139b0 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa538e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1ab4a0 con 0x56502ea0de50 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38812/0 (socket says 172.21.15.31:38812) 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3961137566 learned_addr learned my addr 172.21.15.31:0/3961137566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea139b0 0x56502ea2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9bfff640 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3961137566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 msgr2=0x56502ea0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea0cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3961137566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea139b0 msgr2=0x56502ea2afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.519+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea139b0 0x56502ea2afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3961137566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502ea0a230 con 0x56502ea0e840 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea139b0 0x56502ea2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1f7c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fba840cc6e0 tx=0x7fba84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472aae046963ea66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fba91ffb640 1 -- 172.21.15.31:0/3961137566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84020070 con 0x56502ea0e840 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fba91ffb640 1 -- 172.21.15.31:0/3961137566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba840a40b0 con 0x56502ea0e840 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fba91ffb640 1 -- 172.21.15.31:0/3961137566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84010460 con 0x56502ea0e840 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fbaa538e740 1 -- 172.21.15.31:0/3961137566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 msgr2=0x56502ea1f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1f7c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fba840cc6e0 tx=0x7fba84005880 comp rx=0 tx=0).stop 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fbaa538e740 1 -- 172.21.15.31:0/3961137566 shutdown_connections 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea139b0 0x56502ea2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.520+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1f7c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3961137566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea0de50 0x56502ea0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- 172.21.15.31:0/3961137566 >> 172.21.15.31:0/3961137566 conn(0x56502ea094c0 msgr2=0x56502ea0c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- 172.21.15.31:0/3961137566 shutdown_connections 2024-04-25T00:38:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- 172.21.15.31:0/3961137566 wait complete. 2024-04-25T00:38:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- start start 2024-04-25T00:38:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 Processor -- start 2024-04-25T00:38:49.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:49 smithi031 ceph-mon[29167]: pgmap v608: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea139b0 0x56502ea1e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1ac240 con 0x56502ea0e840 2024-04-25T00:38:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1abb70 con 0x56502ea0de50 2024-04-25T00:38:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1a8db0 con 0x56502ea139b0 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fbaa538e740 10 client.0 _ll_register_callbacks cb 0x7fba94004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.521+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38816/0 (socket says 172.21.15.31:38816) 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3327448032 learned_addr learned my addr 172.21.15.31:0/3327448032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9bfff640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fbaa0b42640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea139b0 0x56502ea1e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea139b0 msgr2=0x56502ea1e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea139b0 0x56502ea1e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3327448032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 msgr2=0x56502ea1cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea1cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 -- 172.21.15.31:0/3327448032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502ea1e690 con 0x56502ea0e840 2024-04-25T00:38:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9bfff640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba9b7fe640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1d090 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fba84097ec0 tx=0x7fba8409f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b45a3c6e4f18055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba7b7fe640 1 -- 172.21.15.31:0/3327448032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba840b9060 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba7b7fe640 1 -- 172.21.15.31:0/3327448032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fba840b43b0 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba840a40b0 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba7b7fe640 1 -- 172.21.15.31:0/3327448032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba84020040 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fbaa538e740 10 client.16059 fetch_fsmap learned FSMap version 26 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56502ea1fa40 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.522+0000 7fba7b7fe640 1 -- 172.21.15.31:0/3327448032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba840a40b0 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fba7b7fe640 1 -- 172.21.15.31:0/3327448032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba840972d0 con 0x56502ea0e840 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fbaa538e740 10 client.16059 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fbaa538e740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fbaa538e740 20 client.16059 populate_metadata read hostname 'smithi031' 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea93780 0x56502ea95ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fbaa538e740 4 client.16059 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56502e979540 con 0x56502ea93780 2024-04-25T00:38:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fba9bfff640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea93780 0x56502ea95ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fba9bfff640 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea93780 0x56502ea95ba0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7713eaa99cda612 server_cookie=133978eaf77d8a4d in_seq=0 out_seq=0 2024-04-25T00:38:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.523+0000 7fba7b7fe640 10 client.16059 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.539+0000 7fba7b7fe640 1 -- 172.21.15.31:0/3327448032 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56502e979540 con 0x56502ea93780 2024-04-25T00:38:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.539+0000 7fba7b7fe640 10 client.16059 handle_command_reply: tid=0 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.539+0000 7fbaa538e740 1 client.16059 shutdown 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 msgr2=0x56502ea1d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1d090 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fba84097ec0 tx=0x7fba8409f8b0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 shutdown_connections 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56502ea93780 0x56502ea95ba0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56502ea139b0 0x56502ea1e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56502ea0e840 0x56502ea1d090 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 --2- 172.21.15.31:0/3327448032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56502ea0de50 0x56502ea1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fba7b7fe640 0 client.16059 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 >> 172.21.15.31:0/3327448032 conn(0x56502ea094c0 msgr2=0x56502ea07bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 shutdown_connections 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 1 -- 172.21.15.31:0/3327448032 wait complete. 2024-04-25T00:38:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa538e740 20 client.16059 trim_cache size 0 max 16384 2024-04-25T00:38:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:38:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:38:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:38:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:38:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fba880a2c30 msgr2=0x7fba880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fba880a2c30 0x7fba880a50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56502ea0f0a0 tx=0x7fba94005df0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 msgr2=0x7fba9c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.540+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 0x7fba9c199fd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d950 tx=0x7fba8400de10 comp rx=0 tx=0).stop 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 shutdown_connections 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba9c102300 0x7fba9c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fba880a2c30 0x7fba880a50f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba9c101940 0x7fba9c199fd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 --2- 172.21.15.31:0/2816760720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c100750 0x7fba9c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 >> 172.21.15.31:0/2816760720 conn(0x7fba9c0fbcc0 msgr2=0x7fba9c109550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 shutdown_connections 2024-04-25T00:38:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:38:49.541+0000 7fbaa25a5640 1 -- 172.21.15.31:0/2816760720 wait complete. 2024-04-25T00:38:49.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:49 smithi169 ceph-mon[33112]: pgmap v608: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:49.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:49 smithi040 ceph-mon[32945]: pgmap v608: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:38:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:51 smithi031 ceph-mon[29167]: pgmap v609: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:38:51.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:51 smithi169 ceph-mon[33112]: pgmap v609: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:38:51.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:51 smithi040 ceph-mon[32945]: pgmap v609: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:38:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:53 smithi031 ceph-mon[29167]: pgmap v610: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:38:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:53 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:38:53.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:53 smithi169 ceph-mon[33112]: pgmap v610: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:38:53.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:53 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:38:53.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:53 smithi040 ceph-mon[32945]: pgmap v610: 97 pgs: 97 active+clean; 1.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:38:53.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:53 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:38:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:38:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:38:54.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:38:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:38:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:38:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:38:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:38:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:38:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:38:55.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:55 smithi031 ceph-mon[29167]: pgmap v611: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:55.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:55 smithi169 ceph-mon[33112]: pgmap v611: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:55.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:55 smithi040 ceph-mon[32945]: pgmap v611: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:38:57.442 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:57 smithi169 ceph-mon[33112]: pgmap v612: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:57.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:57 smithi031 ceph-mon[29167]: pgmap v612: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:57.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:57 smithi040 ceph-mon[32945]: pgmap v612: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:38:59.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:38:59 smithi040 ceph-mon[32945]: pgmap v613: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:38:59.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:38:59 smithi031 ceph-mon[29167]: pgmap v613: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:38:59.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:38:59 smithi169 ceph-mon[33112]: pgmap v613: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:39:01.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:01 smithi031 ceph-mon[29167]: pgmap v614: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T00:39:01.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:01 smithi169 ceph-mon[33112]: pgmap v614: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T00:39:01.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:01 smithi040 ceph-mon[32945]: pgmap v614: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 -- 172.21.15.31:0/1664913481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8102390 msgr2=0x7f25f8104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/1664913481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8102390 0x7f25f8104780 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009900 tx=0x7f25e802a410 comp rx=0 tx=0).stop 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 -- 172.21.15.31:0/1664913481 shutdown_connections 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/1664913481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f81075f0 0x7f25f81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/1664913481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f8104cc0 0x7f25f81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/1664913481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8102390 0x7f25f8104780 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 -- 172.21.15.31:0/1664913481 >> 172.21.15.31:0/1664913481 conn(0x7f25f80fbcc0 msgr2=0x7f25f80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 -- 172.21.15.31:0/1664913481 shutdown_connections 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 -- 172.21.15.31:0/1664913481 wait complete. 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 Processor -- start 2024-04-25T00:39:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.640+0000 7f25fe7b6640 1 -- start start 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fe7b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f8102390 0x7f25f8197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fe7b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 0x7f25f8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fe7b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f81075f0 0x7f25f819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fe7b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810d360 con 0x7f25f8104cc0 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fe7b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810d1e0 con 0x7f25f81075f0 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fe7b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810d4e0 con 0x7f25f8102390 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 0x7f25f8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 0x7f25f8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46818/0 (socket says 172.21.15.31:46818) 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 -- 172.21.15.31:0/3611651125 learned_addr learned my addr 172.21.15.31:0/3611651125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f7fff640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f8102390 0x7f25f8197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f81075f0 0x7f25f819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 -- 172.21.15.31:0/3611651125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f8102390 msgr2=0x7f25f8197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f8102390 0x7f25f8197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 -- 172.21.15.31:0/3611651125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f81075f0 msgr2=0x7f25f819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f81075f0 0x7f25f819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 -- 172.21.15.31:0/3611651125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f819f550 con 0x7f25f8104cc0 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f81075f0 0x7f25f819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f7fff640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f8102390 0x7f25f8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.641+0000 7f25f77fe640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 0x7f25f8197e70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e9c0 tx=0x7f25e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.642+0000 7f25f57fa640 1 -- 172.21.15.31:0/3611651125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000cd80 con 0x7f25f8104cc0 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.642+0000 7f25f57fa640 1 -- 172.21.15.31:0/3611651125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e00054d0 con 0x7f25f8104cc0 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.642+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f819f840 con 0x7f25f8104cc0 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.642+0000 7f25f57fa640 1 -- 172.21.15.31:0/3611651125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0010430 con 0x7f25f8104cc0 2024-04-25T00:39:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.642+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f819fd90 con 0x7f25f8104cc0 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f25f57fa640 1 -- 172.21.15.31:0/3611651125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f25e0004fd0 con 0x7f25f8104cc0 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 Processor -- start 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 -- start start 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 0x55c2f2b76340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b769b0 0x55c2f2b78e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2ad4cb0 con 0x55c2f2b71840 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2a7d5b0 con 0x55c2f2b769b0 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.644+0000 7f260159b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2ad0c70 con 0x55c2f2b70e50 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 0x55c2f2b76340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 0x55c2f2b76340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46824/0 (socket says 172.21.15.31:46824) 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 -- 172.21.15.31:0/1435038423 learned_addr learned my addr 172.21.15.31:0/1435038423 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25fcd53640 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f57fa640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f25e40a2c30 0x7f25e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f77fe640 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b769b0 0x55c2f2b78e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 -- 172.21.15.31:0/1435038423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 msgr2=0x55c2f2b71270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f25e40a2c30 0x7f25e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b71270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 -- 172.21.15.31:0/1435038423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b769b0 msgr2=0x55c2f2b78e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b769b0 0x55c2f2b78e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 -- 172.21.15.31:0/1435038423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f27e9210 con 0x55c2f2b71840 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f77fe640 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b769b0 0x55c2f2b78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f57fa640 1 -- 172.21.15.31:0/3611651125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f25e00cf5e0 con 0x7f25f8104cc0 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25f7fff640 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 0x55c2f2b76340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009900 tx=0x7f25e8004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0dc36384f9159a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.645+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f25e40a2c30 0x7f25e40a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55c2f2b720a0 tx=0x7f25f0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f25ee7fc640 1 -- 172.21.15.31:0/1435038423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8035030 con 0x55c2f2b71840 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f25ee7fc640 1 -- 172.21.15.31:0/1435038423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e80054f0 con 0x55c2f2b71840 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f25ee7fc640 1 -- 172.21.15.31:0/1435038423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e80035f0 con 0x55c2f2b71840 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 -- 172.21.15.31:0/1435038423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 msgr2=0x55c2f2b76340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 0x55c2f2b76340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009900 tx=0x7f25e8004140 comp rx=0 tx=0).stop 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 -- 172.21.15.31:0/1435038423 shutdown_connections 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b769b0 0x55c2f2b78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b71840 0x55c2f2b76340 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 --2- 172.21.15.31:0/1435038423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b71270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 -- 172.21.15.31:0/1435038423 >> 172.21.15.31:0/1435038423 conn(0x55c2f2b6c4c0 msgr2=0x55c2f2b6c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 -- 172.21.15.31:0/1435038423 shutdown_connections 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.646+0000 7f260159b740 1 -- 172.21.15.31:0/1435038423 wait complete. 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 -- start start 2024-04-25T00:39:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 Processor -- start 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b769b0 0x55c2f2b827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2b79900 con 0x55c2f2b70e50 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2a90b50 con 0x55c2f2b71840 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2b79a80 con 0x55c2f2b769b0 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f260159b740 10 client.0 _ll_register_callbacks cb 0x55c2f2acbca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f25f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f25f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46834/0 (socket says 172.21.15.31:46834) 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f25f7fff640 1 -- 172.21.15.31:0/1877659950 learned_addr learned my addr 172.21.15.31:0/1877659950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.647+0000 7f25fcd53640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b769b0 0x55c2f2b827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f77fe640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f7fff640 1 -- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b769b0 msgr2=0x55c2f2b827d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f7fff640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b769b0 0x55c2f2b827d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f7fff640 1 -- 172.21.15.31:0/1877659950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 msgr2=0x55c2f2b7b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f7fff640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f7fff640 1 -- 172.21.15.31:0/1877659950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f28b5620 con 0x55c2f2b70e50 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25f7fff640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7b250 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e8004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8815de810730ab6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25d77fe640 1 -- 172.21.15.31:0/1877659950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8035050 con 0x55c2f2b70e50 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25d77fe640 1 -- 172.21.15.31:0/1877659950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f25e8005010 con 0x55c2f2b70e50 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e80054f0 con 0x55c2f2b70e50 2024-04-25T00:39:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25d77fe640 1 -- 172.21.15.31:0/1877659950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f25e8005210 con 0x55c2f2b70e50 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f260159b740 10 client.16071 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2f2b85ab0 con 0x55c2f2b70e50 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.648+0000 7f25d77fe640 1 -- 172.21.15.31:0/1877659950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e80054f0 con 0x55c2f2b70e50 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f25d77fe640 1 -- 172.21.15.31:0/1877659950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f25e8031880 con 0x55c2f2b70e50 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f260159b740 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f260159b740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f260159b740 20 client.16071 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f260159b740 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2b86e00 0x55c2f2b891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f260159b740 4 client.16071 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f2ae21f0 con 0x55c2f2b86e00 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f25f77fe640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2b86e00 0x55c2f2b891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f25d77fe640 10 client.16071 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.649+0000 7f25f77fe640 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2b86e00 0x55c2f2b891f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2408bc27e2cf422d server_cookie=f7d4ea6997838f58 in_seq=0 out_seq=0 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f25d77fe640 1 -- 172.21.15.31:0/1877659950 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2f2ae21f0 con 0x55c2f2b86e00 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f25d77fe640 10 client.16071 handle_command_reply: tid=0 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 client.16071 shutdown 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 msgr2=0x55c2f2b7b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7b250 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e8004560 comp rx=0 tx=0).stop 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 shutdown_connections 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2b86e00 0x55c2f2b891f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b769b0 0x55c2f2b827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 --2- 172.21.15.31:0/1877659950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7b250 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f25d77fe640 0 client.16071 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.650+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 >> 172.21.15.31:0/1877659950 conn(0x55c2f2b6c4c0 msgr2=0x55c2f2b6b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.651+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 shutdown_connections 2024-04-25T00:39:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.651+0000 7f260159b740 1 -- 172.21.15.31:0/1877659950 wait complete. 2024-04-25T00:39:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.651+0000 7f260159b740 20 client.16071 trim_cache size 0 max 16384 2024-04-25T00:39:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 Processor -- start 2024-04-25T00:39:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 -- start start 2024-04-25T00:39:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b70e50 0x55c2f2b6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b71840 0x55c2f2b82790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2b7a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81a8b80 con 0x55c2f2b769b0 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81aa6c0 con 0x55c2f2b70e50 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f260159b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81a9250 con 0x55c2f2b71840 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2b7a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2b7a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46842/0 (socket says 172.21.15.31:46842) 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 -- 172.21.15.31:0/2010918205 learned_addr learned my addr 172.21.15.31:0/2010918205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25f77fe640 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b71840 0x55c2f2b82790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25f7fff640 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b70e50 0x55c2f2b6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 -- 172.21.15.31:0/2010918205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b71840 msgr2=0x55c2f2b82790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b71840 0x55c2f2b82790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 -- 172.21.15.31:0/2010918205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b70e50 msgr2=0x55c2f2b6c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b70e50 0x55c2f2b6c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25fcd53640 1 -- 172.21.15.31:0/2010918205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f2b897d0 con 0x55c2f2b769b0 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.666+0000 7f25f7fff640 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b70e50 0x55c2f2b6c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f25fcd53640 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2b7a540 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f25f0012ba0 tx=0x7f25f0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f524cdb2b3d2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f25edffb640 1 -- 172.21.15.31:0/2010918205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f001a070 con 0x55c2f2b769b0 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f25edffb640 1 -- 172.21.15.31:0/2010918205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25f0011aa0 con 0x55c2f2b769b0 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f25edffb640 1 -- 172.21.15.31:0/2010918205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0005510 con 0x55c2f2b769b0 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 -- 172.21.15.31:0/2010918205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 msgr2=0x55c2f2b7a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2b7a540 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f25f0012ba0 tx=0x7f25f0009900 comp rx=0 tx=0).stop 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 -- 172.21.15.31:0/2010918205 shutdown_connections 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2b7a540 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b71840 0x55c2f2b82790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 --2- 172.21.15.31:0/2010918205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b70e50 0x55c2f2b6c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 -- 172.21.15.31:0/2010918205 >> 172.21.15.31:0/2010918205 conn(0x55c2f2b6c4c0 msgr2=0x55c2f2b6b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.667+0000 7f260159b740 1 -- 172.21.15.31:0/2010918205 shutdown_connections 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 -- 172.21.15.31:0/2010918205 wait complete. 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 -- start start 2024-04-25T00:39:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 Processor -- start 2024-04-25T00:39:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2beb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81a9ff0 con 0x55c2f2b769b0 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81a9920 con 0x55c2f2b71840 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81a6b60 con 0x55c2f2b70e50 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f260159b740 10 client.0 _ll_register_callbacks cb 0x7f25f0009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f25fcd53640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2beb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f25fcd53640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2beb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46844/0 (socket says 172.21.15.31:46844) 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.668+0000 7f25fcd53640 1 -- 172.21.15.31:0/3313749269 learned_addr learned my addr 172.21.15.31:0/3313749269 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25f77fe640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25f7fff640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25fcd53640 1 -- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 msgr2=0x55c2f2b7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25fcd53640 1 -- 172.21.15.31:0/3313749269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 msgr2=0x55c2f2b7fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25fcd53640 1 -- 172.21.15.31:0/3313749269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f2beb7e0 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25f77fe640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25fcd53640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2beb2a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006010 tx=0x7f25f0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f163f0dfeb317a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25d77fe640 1 -- 172.21.15.31:0/3313749269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f001a040 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25d77fe640 1 -- 172.21.15.31:0/3313749269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f25f0004b20 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f0011aa0 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25d77fe640 1 -- 172.21.15.31:0/3313749269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f25f003b030 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f260159b740 10 client.16077 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2f2beca30 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.669+0000 7f25d77fe640 1 -- 172.21.15.31:0/3313749269 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25f0011aa0 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f25d77fe640 1 -- 172.21.15.31:0/3313749269 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f25f0029020 con 0x55c2f2b769b0 2024-04-25T00:39:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f260159b740 10 client.16077 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f260159b740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f260159b740 20 client.16077 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f260159b740 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2bed3e0 0x55c2f2bef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f260159b740 4 client.16077 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2f2adc540 con 0x55c2f2bed3e0 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f25f7fff640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2bed3e0 0x55c2f2bef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f25f7fff640 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2bed3e0 0x55c2f2bef800 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f395d23593e3eff5 server_cookie=51999cfab1bcea35 in_seq=0 out_seq=0 2024-04-25T00:39:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.670+0000 7f25d77fe640 10 client.16077 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f25d77fe640 1 -- 172.21.15.31:0/3313749269 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2f2adc540 con 0x55c2f2bed3e0 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f25d77fe640 10 client.16077 handle_command_reply: tid=0 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 client.16077 shutdown 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 msgr2=0x55c2f2beb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2beb2a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006010 tx=0x7f25f0005bd0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 shutdown_connections 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c2f2bed3e0 0x55c2f2bef800 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2f2b769b0 0x55c2f2beb2a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c2f2b71840 0x55c2f2b7fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f25d77fe640 0 client.16077 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 --2- 172.21.15.31:0/3313749269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c2f2b70e50 0x55c2f2b7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 >> 172.21.15.31:0/3313749269 conn(0x55c2f2b6c4c0 msgr2=0x55c2f2b6b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 shutdown_connections 2024-04-25T00:39:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 1 -- 172.21.15.31:0/3313749269 wait complete. 2024-04-25T00:39:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.687+0000 7f260159b740 20 client.16077 trim_cache size 0 max 16384 2024-04-25T00:39:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:39:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:39:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:39:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:39:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f25e40a2c30 msgr2=0x7f25e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f25e40a2c30 0x7f25e40a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55c2f2b720a0 tx=0x7f25f0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 msgr2=0x7f25f8197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 0x7f25f8197e70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e9c0 tx=0x7f25e000ee80 comp rx=0 tx=0).stop 2024-04-25T00:39:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 shutdown_connections 2024-04-25T00:39:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f81075f0 0x7f25f819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f25e40a2c30 0x7f25e40a50f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8104cc0 0x7f25f8197e70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 --2- 172.21.15.31:0/3611651125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f8102390 0x7f25f8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 >> 172.21.15.31:0/3611651125 conn(0x7f25f80fbcc0 msgr2=0x7f25f8102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 shutdown_connections 2024-04-25T00:39:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:02.688+0000 7f25fe7b6640 1 -- 172.21.15.31:0/3611651125 wait complete. 2024-04-25T00:39:03.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:03 smithi031 ceph-mon[29167]: pgmap v615: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:39:03.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:03 smithi169 ceph-mon[33112]: pgmap v615: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:39:03.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:03 smithi040 ceph-mon[32945]: pgmap v615: 97 pgs: 97 active+clean; 1.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:39:05.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:05 smithi031 ceph-mon[29167]: pgmap v616: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:39:05.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:05 smithi169 ceph-mon[33112]: pgmap v616: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:39:05.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:05 smithi040 ceph-mon[32945]: pgmap v616: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 22 op/s 2024-04-25T00:39:07.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:07 smithi031 ceph-mon[29167]: pgmap v617: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T00:39:07.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:07 smithi169 ceph-mon[33112]: pgmap v617: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T00:39:07.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:07 smithi040 ceph-mon[32945]: pgmap v617: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T00:39:07.992 INFO:tasks.workunit.client.0.smithi031.stdout:375/600: open fd = 3 2024-04-25T00:39:07.992 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.992 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.992 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:376/600: open fd = 3 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:377/600: open fd = 3 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:378/600: open fd = 3 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:379/600: open fd = 3 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:380/600: open fd = 3 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:381/600: open fd = 3 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:382/600: open fd = 3 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:383/600: open fd = 3 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:384/600: open fd = 3 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.995 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:385/600: open fd = 3 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:386/600: open fd = 3 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:387/600: open fd = 3 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:388/600: open fd = 3 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.996 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:389/600: open fd = 3 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:390/600: open fd = 3 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:391/600: open fd = 3 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:392/600: open fd = 3 2024-04-25T00:39:07.997 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:393/600: open fd = 3 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:394/600: open fd = 3 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.998 INFO:tasks.workunit.client.0.smithi031.stdout:395/600: open fd = 3 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:396/600: open fd = 3 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:397/600: open fd = 3 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:398/600: open fd = 3 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:07.999 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:399/600: open fd = 3 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:400/600: open fd = 3 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:401/600: open fd = 3 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:402/600: open fd = 3 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:403/600: open fd = 3 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:404/600: open fd = 3 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:405/600: open fd = 3 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:406/600: open fd = 3 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.001 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:407/600: open fd = 3 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:408/600: open fd = 3 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:409/600: open fd = 3 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.002 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:410/600: open fd = 3 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:411/600: open fd = 3 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:412/600: open fd = 3 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:413/600: open fd = 3 2024-04-25T00:39:08.003 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:414/600: open fd = 3 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:415/600: open fd = 3 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:416/600: open fd = 3 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.004 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:417/600: open fd = 3 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:418/600: open fd = 3 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:419/600: open fd = 3 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:420/600: open fd = 3 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.005 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:421/600: open fd = 3 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:422/600: open fd = 3 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:423/600: open fd = 3 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.006 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:424/600: open fd = 3 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:425/600: open fd = 3 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:426/600: open fd = 3 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:427/600: open fd = 3 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.007 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:428/600: open fd = 3 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:429/600: open fd = 3 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:430/600: open fd = 3 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.008 INFO:tasks.workunit.client.0.smithi031.stdout:431/600: open fd = 3 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:432/600: open fd = 3 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:433/600: open fd = 3 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:434/600: open fd = 3 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.009 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:435/600: open fd = 3 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:436/600: open fd = 3 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:39:08.010 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:39:09.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:09 smithi031 ceph-mon[29167]: pgmap v618: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T00:39:09.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:09 smithi169 ceph-mon[33112]: pgmap v618: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T00:39:09.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:09 smithi040 ceph-mon[32945]: pgmap v618: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T00:39:11.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:11 smithi031 ceph-mon[29167]: pgmap v619: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:39:11.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:11 smithi169 ceph-mon[33112]: pgmap v619: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:39:11.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:11 smithi040 ceph-mon[32945]: pgmap v619: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:39:13.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:13 smithi031 ceph-mon[29167]: pgmap v620: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:13.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:13 smithi169 ceph-mon[33112]: pgmap v620: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:13.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:13 smithi040 ceph-mon[32945]: pgmap v620: 97 pgs: 97 active+clean; 1.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:15.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:15 smithi031 ceph-mon[29167]: pgmap v621: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:39:15.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:15 smithi169 ceph-mon[33112]: pgmap v621: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:39:15.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:15 smithi040 ceph-mon[32945]: pgmap v621: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-25T00:39:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 -- 172.21.15.31:0/132087557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 msgr2=0x7f7dfc100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/132087557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc100b70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009900 tx=0x7f7dec02a410 comp rx=0 tx=0).stop 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 -- 172.21.15.31:0/132087557 shutdown_connections 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/132087557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 0x7f7dfc108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/132087557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dfc101940 0x7f7dfc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/132087557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc100b70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 -- 172.21.15.31:0/132087557 >> 172.21.15.31:0/132087557 conn(0x7f7dfc0fbcc0 msgr2=0x7f7dfc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 -- 172.21.15.31:0/132087557 shutdown_connections 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.787+0000 7f7e00aa3640 1 -- 172.21.15.31:0/132087557 wait complete. 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 Processor -- start 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 -- start start 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc10b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dfc101940 0x7f7dfc109710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 0x7f7dfc109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc10d330 con 0x7f7dfc100750 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc10d1b0 con 0x7f7dfc102300 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7e00aa3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc10d4b0 con 0x7f7dfc101940 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc10b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc10b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40096/0 (socket says 172.21.15.31:40096) 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.788+0000 7f7dfa59c640 1 -- 172.21.15.31:0/3288458641 learned_addr learned my addr 172.21.15.31:0/3288458641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 0x7f7dfc109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7df3fff640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dfc101940 0x7f7dfc109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfa59c640 1 -- 172.21.15.31:0/3288458641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dfc101940 msgr2=0x7f7dfc109710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfa59c640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dfc101940 0x7f7dfc109710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfa59c640 1 -- 172.21.15.31:0/3288458641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 msgr2=0x7f7dfc109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfa59c640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 0x7f7dfc109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfa59c640 1 -- 172.21.15.31:0/3288458641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc10a4e0 con 0x7f7dfc100750 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 0x7f7dfc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7dfa59c640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc10b0d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0314e0 tx=0x7f7dec031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7df37fe640 1 -- 172.21.15.31:0/3288458641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec03f030 con 0x7f7dfc100750 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7df37fe640 1 -- 172.21.15.31:0/3288458641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dec0044b0 con 0x7f7dfc100750 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc1a19f0 con 0x7f7dfc100750 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7df37fe640 1 -- 172.21.15.31:0/3288458641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec035070 con 0x7f7dfc100750 2024-04-25T00:39:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.789+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a1f90 con 0x7f7dfc100750 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.791+0000 7f7df37fe640 1 -- 172.21.15.31:0/3288458641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7dec042030 con 0x7f7dfc100750 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 Processor -- start 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 -- start start 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c509270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7df37fe640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7de80a2cb0 0x7f7de80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c50e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c510e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555c7c46ccb0 con 0x555c7c50e9b0 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555c7c4155b0 con 0x555c7c508e50 2024-04-25T00:39:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.792+0000 7f7e037a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555c7c468c70 con 0x555c7c509840 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfa59c640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7de80a2cb0 0x7f7de80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c510e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c510e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40100/0 (socket says 172.21.15.31:40100) 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 -- 172.21.15.31:0/3395961794 learned_addr learned my addr 172.21.15.31:0/3395961794 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7df3fff640 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c50e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c509270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7df37fe640 1 -- 172.21.15.31:0/3288458641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7dec0ef6a0 con 0x7f7dfc100750 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 -- 172.21.15.31:0/3395961794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 msgr2=0x555c7c50e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c50e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 -- 172.21.15.31:0/3395961794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 msgr2=0x555c7c509270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c509270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 -- 172.21.15.31:0/3395961794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7c181210 con 0x555c7c50e9b0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c510e40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009fd0 tx=0x7f7df4009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5ac95e514cf3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7dfa59c640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7de80a2cb0 0x7f7de80a5170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0a95f0 tx=0x7f7dec0b6820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7df0ff9640 1 -- 172.21.15.31:0/3395961794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4010aa0 con 0x555c7c50e9b0 2024-04-25T00:39:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7df0ff9640 1 -- 172.21.15.31:0/3395961794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df4010c40 con 0x555c7c50e9b0 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.793+0000 7f7df0ff9640 1 -- 172.21.15.31:0/3395961794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df400e680 con 0x555c7c50e9b0 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 -- 172.21.15.31:0/3395961794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 msgr2=0x555c7c510e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c510e40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009fd0 tx=0x7f7df4009b90 comp rx=0 tx=0).stop 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 -- 172.21.15.31:0/3395961794 shutdown_connections 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c510e40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c50e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 --2- 172.21.15.31:0/3395961794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c509270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 -- 172.21.15.31:0/3395961794 >> 172.21.15.31:0/3395961794 conn(0x555c7c5044c0 msgr2=0x555c7c5048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 -- 172.21.15.31:0/3395961794 shutdown_connections 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.794+0000 7f7e037a8740 1 -- 172.21.15.31:0/3395961794 wait complete. 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 -- start start 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 Processor -- start 2024-04-25T00:39:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c513330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c513870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c51a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555c7c511960 con 0x555c7c50e9b0 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555c7c428b50 con 0x555c7c508e50 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555c7c511ae0 con 0x555c7c509840 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7e037a8740 10 client.0 _ll_register_callbacks cb 0x555c7c463ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c51a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c51a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40116/0 (socket says 172.21.15.31:40116) 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 -- 172.21.15.31:0/2589295162 learned_addr learned my addr 172.21.15.31:0/2589295162 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7df3fff640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c513870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c513330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 -- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 msgr2=0x555c7c513870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c513870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 -- 172.21.15.31:0/2589295162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 msgr2=0x555c7c513330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c513330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7dfad9d640 1 -- 172.21.15.31:0/2589295162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7c24d620 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.795+0000 7f7df3fff640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c513870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c51a8b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7df4002230 tx=0x7f7df401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b7ec08c4f3e211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7de1ffb640 1 -- 172.21.15.31:0/2589295162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4026040 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7de1ffb640 1 -- 172.21.15.31:0/2589295162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7df401e640 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df4010c40 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7de1ffb640 1 -- 172.21.15.31:0/2589295162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7df4033030 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 10 client.16089 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c7c51dc20 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7de1ffb640 1 -- 172.21.15.31:0/2589295162 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df4010c40 con 0x555c7c50e9b0 2024-04-25T00:39:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7de1ffb640 1 -- 172.21.15.31:0/2589295162 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7df402c030 con 0x555c7c50e9b0 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 10 client.16089 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 20 client.16089 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.796+0000 7f7e037a8740 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c51f070 0x555c7c521460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.797+0000 7f7e037a8740 4 client.16089 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.797+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c7c47a1f0 con 0x555c7c51f070 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.797+0000 7f7df3fff640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c51f070 0x555c7c521460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.797+0000 7f7df3fff640 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c51f070 0x555c7c521460 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa7f8ea68efb958 server_cookie=c6d7858f43a824c9 in_seq=0 out_seq=0 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.797+0000 7f7de1ffb640 10 client.16089 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7de1ffb640 1 -- 172.21.15.31:0/2589295162 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c7c47a1f0 con 0x555c7c51f070 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7de1ffb640 10 client.16089 handle_command_reply: tid=0 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 client.16089 shutdown 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 msgr2=0x555c7c51a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c51a8b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7df4002230 tx=0x7f7df401ec00 comp rx=0 tx=0).stop 2024-04-25T00:39:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 shutdown_connections 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c51f070 0x555c7c521460 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c51a8b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c513870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 --2- 172.21.15.31:0/2589295162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c513330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7de1ffb640 0 client.16089 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 >> 172.21.15.31:0/2589295162 conn(0x555c7c5044c0 msgr2=0x555c7c5035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 shutdown_connections 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.798+0000 7f7e037a8740 1 -- 172.21.15.31:0/2589295162 wait complete. 2024-04-25T00:39:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.799+0000 7f7e037a8740 20 client.16089 trim_cache size 0 max 16384 2024-04-25T00:39:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 Processor -- start 2024-04-25T00:39:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 -- start start 2024-04-25T00:39:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c502d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c51a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c503270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1aad80 con 0x555c7c50e9b0 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1ac8c0 con 0x555c7c508e50 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7e037a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1ab450 con 0x555c7c509840 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c503270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c503270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40122/0 (socket says 172.21.15.31:40122) 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 -- 172.21.15.31:0/1732956867 learned_addr learned my addr 172.21.15.31:0/1732956867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7df3fff640 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c51a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c502d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 -- 172.21.15.31:0/1732956867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 msgr2=0x555c7c51a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c51a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 -- 172.21.15.31:0/1732956867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 msgr2=0x555c7c502d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c502d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 -- 172.21.15.31:0/1732956867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7c521ac0 con 0x555c7c50e9b0 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.814+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c503270 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7df4011010 tx=0x7f7df4012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31667c97b9624043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7de3fff640 1 -- 172.21.15.31:0/1732956867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4005bf0 con 0x555c7c50e9b0 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7de3fff640 1 -- 172.21.15.31:0/1732956867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df4009500 con 0x555c7c50e9b0 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7de3fff640 1 -- 172.21.15.31:0/1732956867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4038430 con 0x555c7c50e9b0 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 -- 172.21.15.31:0/1732956867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 msgr2=0x555c7c503270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c503270 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7df4011010 tx=0x7f7df4012730 comp rx=0 tx=0).stop 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 -- 172.21.15.31:0/1732956867 shutdown_connections 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c50e9b0 0x555c7c503270 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c509840 0x555c7c51a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1732956867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c508e50 0x555c7c502d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 -- 172.21.15.31:0/1732956867 >> 172.21.15.31:0/1732956867 conn(0x555c7c5044c0 msgr2=0x555c7c5048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 -- 172.21.15.31:0/1732956867 shutdown_connections 2024-04-25T00:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.815+0000 7f7e037a8740 1 -- 172.21.15.31:0/1732956867 wait complete. 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 -- start start 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 Processor -- start 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c508e50 0x555c7c517d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 0x555c7c5182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c50e9b0 0x555c7c5192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1ac1f0 con 0x555c7c509840 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1abb20 con 0x555c7c50e9b0 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1a8d60 con 0x555c7c508e50 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7e037a8740 10 client.0 _ll_register_callbacks cb 0x7f7de400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 0x555c7c5182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 0x555c7c5182c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40126/0 (socket says 172.21.15.31:40126) 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 -- 172.21.15.31:0/1181960160 learned_addr learned my addr 172.21.15.31:0/1181960160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c508e50 0x555c7c517d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c50e9b0 0x555c7c5192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 -- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c508e50 msgr2=0x555c7c517d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c508e50 0x555c7c517d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 -- 172.21.15.31:0/1181960160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c50e9b0 msgr2=0x555c7c5192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c50e9b0 0x555c7c5192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 -- 172.21.15.31:0/1181960160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7c5198c0 con 0x555c7c509840 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c50e9b0 0x555c7c5192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7df3fff640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 0x555c7c5182c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x555c7c509c40 tx=0x7f7de4013990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff07915097bd2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.816+0000 7f7de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7de1ffb640 1 -- 172.21.15.31:0/1181960160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de401d640 con 0x555c7c509840 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7de1ffb640 1 -- 172.21.15.31:0/1181960160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7de401d7e0 con 0x555c7c509840 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df4009500 con 0x555c7c509840 2024-04-25T00:39:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7de1ffb640 1 -- 172.21.15.31:0/1181960160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7de402d050 con 0x555c7c509840 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 10 client.16095 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c7c5855c0 con 0x555c7c509840 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7de1ffb640 1 -- 172.21.15.31:0/1181960160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df4009500 con 0x555c7c509840 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7de1ffb640 1 -- 172.21.15.31:0/1181960160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7de4004bf0 con 0x555c7c509840 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 10 client.16095 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 20 client.16095 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c585f20 0x555c7c588340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 4 client.16095 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.817+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c7c474540 con 0x555c7c585f20 2024-04-25T00:39:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.818+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c585f20 0x555c7c588340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.818+0000 7f7dfad9d640 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c585f20 0x555c7c588340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428f44a08d8942c4 server_cookie=2d7aa3466700f7d4 in_seq=0 out_seq=0 2024-04-25T00:39:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.818+0000 7f7de1ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7de1ffb640 1 -- 172.21.15.31:0/1181960160 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c7c474540 con 0x555c7c585f20 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7de1ffb640 10 client.16095 handle_command_reply: tid=0 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 client.16095 shutdown 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 msgr2=0x555c7c5182c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 0x555c7c5182c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x555c7c509c40 tx=0x7f7de4013990 comp rx=0 tx=0).stop 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 shutdown_connections 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x555c7c585f20 0x555c7c588340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c7c50e9b0 0x555c7c5192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c7c509840 0x555c7c5182c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 --2- 172.21.15.31:0/1181960160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7c508e50 0x555c7c517d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7de1ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 >> 172.21.15.31:0/1181960160 conn(0x555c7c5044c0 msgr2=0x555c7c502d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 shutdown_connections 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 1 -- 172.21.15.31:0/1181960160 wait complete. 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.835+0000 7f7e037a8740 20 client.16095 trim_cache size 0 max 16384 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:39:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:39:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7de80a2cb0 msgr2=0x7f7de80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7de80a2cb0 0x7f7de80a5170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0a95f0 tx=0x7f7dec0b6820 comp rx=0 tx=0).stop 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 msgr2=0x7f7dfc10b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc10b0d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0314e0 tx=0x7f7dec031a30 comp rx=0 tx=0).stop 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 shutdown_connections 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc102300 0x7f7dfc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dfc101940 0x7f7dfc109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f7de80a2cb0 0x7f7de80a5170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 --2- 172.21.15.31:0/3288458641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dfc100750 0x7f7dfc10b0d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 >> 172.21.15.31:0/3288458641 conn(0x7f7dfc0fbcc0 msgr2=0x7f7dfc104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 shutdown_connections 2024-04-25T00:39:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:15.836+0000 7f7e00aa3640 1 -- 172.21.15.31:0/3288458641 wait complete. 2024-04-25T00:39:17.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:17 smithi031 ceph-mon[29167]: pgmap v622: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:17.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:17 smithi169 ceph-mon[33112]: pgmap v622: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:17.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:17 smithi040 ceph-mon[32945]: pgmap v622: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:19.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:19 smithi031 ceph-mon[29167]: pgmap v623: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-25T00:39:19.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:19 smithi169 ceph-mon[33112]: pgmap v623: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-25T00:39:19.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:19 smithi040 ceph-mon[32945]: pgmap v623: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-25T00:39:21.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:21 smithi031 ceph-mon[29167]: pgmap v624: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-04-25T00:39:21.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:21 smithi169 ceph-mon[33112]: pgmap v624: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-04-25T00:39:21.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:21 smithi040 ceph-mon[32945]: pgmap v624: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-04-25T00:39:23.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:23 smithi031 ceph-mon[29167]: pgmap v625: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:23.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:23 smithi169 ceph-mon[33112]: pgmap v625: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:23.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:23 smithi040 ceph-mon[32945]: pgmap v625: 97 pgs: 97 active+clean; 1.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:25.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:25 smithi031 ceph-mon[29167]: pgmap v626: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:25.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:25 smithi169 ceph-mon[33112]: pgmap v626: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:25.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:25 smithi040 ceph-mon[32945]: pgmap v626: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:27.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:27 smithi031 ceph-mon[29167]: pgmap v627: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:27.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:27 smithi169 ceph-mon[33112]: pgmap v627: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:27.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:27 smithi040 ceph-mon[32945]: pgmap v627: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 -- 172.21.15.31:0/520020431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f81017d0 msgr2=0x7f14f8101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 --2- 172.21.15.31:0/520020431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f81017d0 0x7f14f8101bf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec02a410 comp rx=0 tx=0).stop 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 -- 172.21.15.31:0/520020431 shutdown_connections 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 --2- 172.21.15.31:0/520020431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f8103380 0x7f14f8109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 --2- 172.21.15.31:0/520020431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f81029c0 0x7f14f8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 --2- 172.21.15.31:0/520020431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f81017d0 0x7f14f8101bf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 -- 172.21.15.31:0/520020431 >> 172.21.15.31:0/520020431 conn(0x7f14f80fcd60 msgr2=0x7f14f80ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 -- 172.21.15.31:0/520020431 shutdown_connections 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.934+0000 7f14fecda640 1 -- 172.21.15.31:0/520020431 wait complete. 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 Processor -- start 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 -- start start 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f81017d0 0x7f14f8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f81029c0 0x7f14f8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 0x7f14f819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810d340 con 0x7f14f8103380 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810d1c0 con 0x7f14f81029c0 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fecda640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810d4c0 con 0x7f14f81017d0 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fd277640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 0x7f14f819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fd277640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 0x7f14f819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43978/0 (socket says 172.21.15.31:43978) 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fd277640 1 -- 172.21.15.31:0/3212166266 learned_addr learned my addr 172.21.15.31:0/3212166266 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14fca76640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f81017d0 0x7f14f8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.935+0000 7f14f7fff640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f81029c0 0x7f14f8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fd277640 1 -- 172.21.15.31:0/3212166266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f81017d0 msgr2=0x7f14f8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fd277640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f81017d0 0x7f14f8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fd277640 1 -- 172.21.15.31:0/3212166266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f81029c0 msgr2=0x7f14f8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fd277640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f81029c0 0x7f14f8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fd277640 1 -- 172.21.15.31:0/3212166266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f819f5b0 con 0x7f14f8103380 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fd277640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 0x7f14f819ef40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f14f000cad0 tx=0x7f14f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14f5ffb640 1 -- 172.21.15.31:0/3212166266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f000ae20 con 0x7f14f8103380 2024-04-25T00:39:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14f5ffb640 1 -- 172.21.15.31:0/3212166266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14f0005020 con 0x7f14f8103380 2024-04-25T00:39:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f819f8a0 con 0x7f14f8103380 2024-04-25T00:39:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14f5ffb640 1 -- 172.21.15.31:0/3212166266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f000f6b0 con 0x7f14f8103380 2024-04-25T00:39:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.936+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f819fda0 con 0x7f14f8103380 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.938+0000 7f14f5ffb640 1 -- 172.21.15.31:0/3212166266 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f14f00051d0 con 0x7f14f8103380 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 Processor -- start 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 -- start start 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x56446767e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467683340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x564467685e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5644675e1cb0 con 0x56446767e840 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56446758a5b0 con 0x56446767de50 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f15019ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5644675ddc70 con 0x5644676839b0 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f14f5ffb640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14e80a2c30 0x7f14e80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.939+0000 7f14fca76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467683340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467683340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43992/0 (socket says 172.21.15.31:43992) 2024-04-25T00:39:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 -- 172.21.15.31:0/2464326171 learned_addr learned my addr 172.21.15.31:0/2464326171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14f7fff640 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x56446767e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fd277640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14e80a2c30 0x7f14e80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14f7fff640 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x564467685e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 -- 172.21.15.31:0/2464326171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 msgr2=0x564467685e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x564467685e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 -- 172.21.15.31:0/2464326171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 msgr2=0x56446767e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x56446767e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 -- 172.21.15.31:0/2464326171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644672f6210 con 0x56446767e840 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14f5ffb640 1 -- 172.21.15.31:0/3212166266 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f14f00cf0c0 con 0x7f14f8103380 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fca76640 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467683340 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201a50a63ee0e636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14fd277640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14e80a2c30 0x7f14e80a50f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f14f00963e0 tx=0x7f14f00c6420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14e2ffd640 1 -- 172.21.15.31:0/2464326171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec035030 con 0x56446767e840 2024-04-25T00:39:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14e2ffd640 1 -- 172.21.15.31:0/2464326171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14ec040070 con 0x56446767e840 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.940+0000 7f14e2ffd640 1 -- 172.21.15.31:0/2464326171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec031b20 con 0x56446767e840 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 -- 172.21.15.31:0/2464326171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 msgr2=0x564467683340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467683340 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec004390 comp rx=0 tx=0).stop 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 -- 172.21.15.31:0/2464326171 shutdown_connections 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x564467685e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467683340 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 --2- 172.21.15.31:0/2464326171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x56446767e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 -- 172.21.15.31:0/2464326171 >> 172.21.15.31:0/2464326171 conn(0x5644676794c0 msgr2=0x5644676798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 -- 172.21.15.31:0/2464326171 shutdown_connections 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 -- 172.21.15.31:0/2464326171 wait complete. 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.941+0000 7f15019ef740 1 -- start start 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 Processor -- start 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x564467688160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x5644676886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x56446768f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564467686900 con 0x56446767e840 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56446759db50 con 0x56446767de50 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564467686a80 con 0x5644676839b0 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f15019ef740 10 client.0 _ll_register_callbacks cb 0x5644675d8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x5644676886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x5644676886a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44002/0 (socket says 172.21.15.31:44002) 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 -- 172.21.15.31:0/1977611052 learned_addr learned my addr 172.21.15.31:0/1977611052 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14f7fff640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x56446768f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14f7fff640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x564467688160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 -- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 msgr2=0x56446768f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x56446768f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 -- 172.21.15.31:0/1977611052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 msgr2=0x564467688160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x564467688160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14fca76640 1 -- 172.21.15.31:0/1977611052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644673c2620 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.942+0000 7f14f7fff640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x56446768f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14fca76640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x5644676886a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f14ec005ab0 tx=0x7f14ec031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc200471838acf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14d7fff640 1 -- 172.21.15.31:0/1977611052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec035030 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14d7fff640 1 -- 172.21.15.31:0/1977611052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f14ec045880 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec040070 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14d7fff640 1 -- 172.21.15.31:0/1977611052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14ec045a80 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 10 client.16107 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564467692a50 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14d7fff640 1 -- 172.21.15.31:0/1977611052 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14ec040070 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f14d7fff640 1 -- 172.21.15.31:0/1977611052 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f14ec058430 con 0x56446767e840 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 20 client.16107 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.943+0000 7f15019ef740 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564467693da0 0x564467696190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.944+0000 7f15019ef740 4 client.16107 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.944+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644675ef1f0 con 0x564467693da0 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.944+0000 7f14f7fff640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564467693da0 0x564467696190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.944+0000 7f14f7fff640 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564467693da0 0x564467696190 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db37697bb9026332 server_cookie=5d222ecde9d657ad in_seq=0 out_seq=0 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.944+0000 7f14d7fff640 10 client.16107 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f14d7fff640 1 -- 172.21.15.31:0/1977611052 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644675ef1f0 con 0x564467693da0 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f14d7fff640 10 client.16107 handle_command_reply: tid=0 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 client.16107 shutdown 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 msgr2=0x5644676886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x5644676886a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f14ec005ab0 tx=0x7f14ec031ad0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 shutdown_connections 2024-04-25T00:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x564467693da0 0x564467696190 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644676839b0 0x56446768f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x5644676886a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f14d7fff640 0 client.16107 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 --2- 172.21.15.31:0/1977611052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56446767de50 0x564467688160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 >> 172.21.15.31:0/1977611052 conn(0x5644676794c0 msgr2=0x5644676785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 shutdown_connections 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 1 -- 172.21.15.31:0/1977611052 wait complete. 2024-04-25T00:39:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.945+0000 7f15019ef740 20 client.16107 trim_cache size 0 max 16384 2024-04-25T00:39:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 Processor -- start 2024-04-25T00:39:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 -- start start 2024-04-25T00:39:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767de50 0x56446767fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467680140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a8bd0 con 0x56446767e840 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81aa710 con 0x5644676839b0 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f15019ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a92a0 con 0x56446767de50 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467680140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467680140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44014/0 (socket says 172.21.15.31:44014) 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 -- 172.21.15.31:0/956436365 learned_addr learned my addr 172.21.15.31:0/956436365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14f7fff640 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14f7fff640 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767de50 0x56446767fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 -- 172.21.15.31:0/956436365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767de50 msgr2=0x56446767fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767de50 0x56446767fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 -- 172.21.15.31:0/956436365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 msgr2=0x56446768f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14fca76640 1 -- 172.21.15.31:0/956436365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564467696500 con 0x56446767e840 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.961+0000 7f14f7fff640 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767de50 0x56446767fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f14fca76640 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467680140 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f14ec006010 tx=0x7f14ec004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5c7d8592fa5507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f14e27fc640 1 -- 172.21.15.31:0/956436365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec05d070 con 0x56446767e840 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f14e27fc640 1 -- 172.21.15.31:0/956436365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14ec040030 con 0x56446767e840 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f14e27fc640 1 -- 172.21.15.31:0/956436365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec035040 con 0x56446767e840 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 -- 172.21.15.31:0/956436365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 msgr2=0x564467680140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467680140 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f14ec006010 tx=0x7f14ec004390 comp rx=0 tx=0).stop 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 -- 172.21.15.31:0/956436365 shutdown_connections 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767e840 0x564467680140 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 --2- 172.21.15.31:0/956436365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767de50 0x56446767fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 -- 172.21.15.31:0/956436365 >> 172.21.15.31:0/956436365 conn(0x5644676794c0 msgr2=0x5644676798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.962+0000 7f15019ef740 1 -- 172.21.15.31:0/956436365 shutdown_connections 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 -- 172.21.15.31:0/956436365 wait complete. 2024-04-25T00:39:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 -- start start 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 Processor -- start 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 0x56446768c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767e840 0x56446768cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81aa040 con 0x56446767de50 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a9970 con 0x5644676839b0 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a6bb0 con 0x56446767e840 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f15019ef740 10 client.0 _ll_register_callbacks cb 0x7f14ec005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f14f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 0x56446768c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f14f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 0x56446768c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44018/0 (socket says 172.21.15.31:44018) 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f14f7fff640 1 -- 172.21.15.31:0/4202292202 learned_addr learned my addr 172.21.15.31:0/4202292202 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.963+0000 7f14fca76640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767e840 0x56446768cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 -- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767e840 msgr2=0x56446768cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767e840 0x56446768cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 -- 172.21.15.31:0/4202292202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 msgr2=0x56446768def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 -- 172.21.15.31:0/4202292202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446768e4e0 con 0x56446767de50 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14f7fff640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 0x56446768c9a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005d00 tx=0x7f14ec031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78282322c596d49d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14d7fff640 1 -- 172.21.15.31:0/4202292202 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e401b040 con 0x56446767de50 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14d7fff640 1 -- 172.21.15.31:0/4202292202 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f14e4004250 con 0x56446767de50 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec040030 con 0x56446767de50 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14d7fff640 1 -- 172.21.15.31:0/4202292202 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14e4035050 con 0x56446767de50 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f15019ef740 10 client.16113 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56446768f830 con 0x56446767de50 2024-04-25T00:39:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.964+0000 7f14d7fff640 1 -- 172.21.15.31:0/4202292202 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14ec040030 con 0x56446767de50 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f14d7fff640 1 -- 172.21.15.31:0/4202292202 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f14e402e060 con 0x56446767de50 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f15019ef740 10 client.16113 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f15019ef740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f15019ef740 20 client.16113 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f15019ef740 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5644676fa920 0x5644676fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f15019ef740 4 client.16113 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644675e9540 con 0x5644676fa920 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f14fca76640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5644676fa920 0x5644676fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f14d7fff640 10 client.16113 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.965+0000 7f14fca76640 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5644676fa920 0x5644676fcd40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d4df09dbab547c server_cookie=a67cb609f13674db in_seq=0 out_seq=0 2024-04-25T00:39:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f14d7fff640 1 -- 172.21.15.31:0/4202292202 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644675e9540 con 0x5644676fa920 2024-04-25T00:39:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f14d7fff640 10 client.16113 handle_command_reply: tid=0 2024-04-25T00:39:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 client.16113 shutdown 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 msgr2=0x56446768c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 0x56446768c9a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005d00 tx=0x7f14ec031580 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 shutdown_connections 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5644676fa920 0x5644676fcd40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5644676839b0 0x56446768def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56446767e840 0x56446768cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f14d7fff640 0 client.16113 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 --2- 172.21.15.31:0/4202292202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56446767de50 0x56446768c9a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 >> 172.21.15.31:0/4202292202 conn(0x5644676794c0 msgr2=0x564467688140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 shutdown_connections 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 1 -- 172.21.15.31:0/4202292202 wait complete. 2024-04-25T00:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.982+0000 7f15019ef740 20 client.16113 trim_cache size 0 max 16384 2024-04-25T00:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14e80a2c30 msgr2=0x7f14e80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14e80a2c30 0x7f14e80a50f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f14f00963e0 tx=0x7f14f00c6420 comp rx=0 tx=0).stop 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 msgr2=0x7f14f819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 0x7f14f819ef40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f14f000cad0 tx=0x7f14f000cf90 comp rx=0 tx=0).stop 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 shutdown_connections 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14e80a2c30 0x7f14e80a50f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f8103380 0x7f14f819ef40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f81029c0 0x7f14f8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 --2- 172.21.15.31:0/3212166266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f81017d0 0x7f14f8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 >> 172.21.15.31:0/3212166266 conn(0x7f14f80fcd60 msgr2=0x7f14f8105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 shutdown_connections 2024-04-25T00:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:28.983+0000 7f14fecda640 1 -- 172.21.15.31:0/3212166266 wait complete. 2024-04-25T00:39:29.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:29 smithi031 ceph-mon[29167]: pgmap v628: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:29.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:29 smithi169 ceph-mon[33112]: pgmap v628: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:29.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:29 smithi040 ceph-mon[32945]: pgmap v628: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:31.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:31 smithi031 ceph-mon[29167]: pgmap v629: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:39:31.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:31 smithi169 ceph-mon[33112]: pgmap v629: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:39:31.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:31 smithi040 ceph-mon[32945]: pgmap v629: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:39:33.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:33 smithi031 ceph-mon[29167]: pgmap v630: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:39:33.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:33 smithi169 ceph-mon[33112]: pgmap v630: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:39:33.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:33 smithi040 ceph-mon[32945]: pgmap v630: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:39:35.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:35 smithi031 ceph-mon[29167]: pgmap v631: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:35.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:35 smithi169 ceph-mon[33112]: pgmap v631: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:35.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:35 smithi040 ceph-mon[32945]: pgmap v631: 97 pgs: 97 active+clean; 2.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:37.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:37 smithi031 ceph-mon[29167]: pgmap v632: 97 pgs: 97 active+clean; 2.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:37.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:37 smithi169 ceph-mon[33112]: pgmap v632: 97 pgs: 97 active+clean; 2.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:37.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:37 smithi040 ceph-mon[32945]: pgmap v632: 97 pgs: 97 active+clean; 2.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:39.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:39 smithi031 ceph-mon[29167]: pgmap v633: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:39.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:39 smithi169 ceph-mon[33112]: pgmap v633: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:39.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:39 smithi040 ceph-mon[32945]: pgmap v633: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:41.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:41 smithi031 ceph-mon[29167]: pgmap v634: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:39:41.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:41 smithi169 ceph-mon[33112]: pgmap v634: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:39:41.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:41 smithi040 ceph-mon[32945]: pgmap v634: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 -- 172.21.15.31:0/2245666464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 msgr2=0x7faff8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 --2- 172.21.15.31:0/2245666464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8101d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009900 tx=0x7fafe802a410 comp rx=0 tx=0).stop 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 -- 172.21.15.31:0/2245666464 shutdown_connections 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 --2- 172.21.15.31:0/2245666464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff8102250 0x7faff8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 --2- 172.21.15.31:0/2245666464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8101d10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 --2- 172.21.15.31:0/2245666464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faff81006a0 0x7faff8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 -- 172.21.15.31:0/2245666464 >> 172.21.15.31:0/2245666464 conn(0x7faff80fbc10 msgr2=0x7faff80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 -- 172.21.15.31:0/2245666464 shutdown_connections 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.083+0000 7faffff63640 1 -- 172.21.15.31:0/2245666464 wait complete. 2024-04-25T00:39:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 Processor -- start 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 -- start start 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faff81006a0 0x7faff8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff8102250 0x7faff819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faff810d2f0 con 0x7faff8101890 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faff810d170 con 0x7faff81006a0 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffff63640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faff810d470 con 0x7faff8102250 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41074/0 (socket says 172.21.15.31:41074) 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 -- 172.21.15.31:0/3258264776 learned_addr learned my addr 172.21.15.31:0/3258264776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffe500640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff8102250 0x7faff819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffdcff640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faff81006a0 0x7faff8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 -- 172.21.15.31:0/3258264776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff8102250 msgr2=0x7faff819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff8102250 0x7faff819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 -- 172.21.15.31:0/3258264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faff81006a0 msgr2=0x7faff8197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faff81006a0 0x7faff8197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.084+0000 7faffd4fe640 1 -- 172.21.15.31:0/3258264776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff819f550 con 0x7faff8101890 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.085+0000 7faffd4fe640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8197ec0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fafe8004770 tx=0x7fafe80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.085+0000 7fafeeffd640 1 -- 172.21.15.31:0/3258264776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe803f030 con 0x7faff8101890 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.085+0000 7fafeeffd640 1 -- 172.21.15.31:0/3258264776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafe8003cb0 con 0x7faff8101890 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.085+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff819f7e0 con 0x7faff8101890 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.085+0000 7fafeeffd640 1 -- 172.21.15.31:0/3258264776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe803f030 con 0x7faff8101890 2024-04-25T00:39:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.085+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff819fcd0 con 0x7faff8101890 2024-04-25T00:39:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fafeeffd640 1 -- 172.21.15.31:0/3258264776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fafe8042030 con 0x7faff8101890 2024-04-25T00:39:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 Processor -- start 2024-04-25T00:39:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 -- start start 2024-04-25T00:39:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d2b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d30340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d32e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a8a1c8ecb0 con 0x55a8a1d2ae50 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a8a1c375b0 con 0x55a8a1d309b0 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fb002c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a8a1c8ac70 con 0x55a8a1d2b840 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.088+0000 7fafeeffd640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fafe40a2e40 0x7fafe40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d2b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d2b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41084/0 (socket says 172.21.15.31:41084) 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 -- 172.21.15.31:0/2444763522 learned_addr learned my addr 172.21.15.31:0/2444763522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fafe40a2e40 0x7fafe40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffdcff640 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d30340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7fafeeffd640 1 -- 172.21.15.31:0/3258264776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fafe80ef620 con 0x7faff8101890 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffd4fe640 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d32e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 -- 172.21.15.31:0/2444763522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 msgr2=0x55a8a1d30340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d30340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 -- 172.21.15.31:0/2444763522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 msgr2=0x55a8a1d32e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d32e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 -- 172.21.15.31:0/2444763522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a19a3210 con 0x55a8a1d2ae50 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffdcff640 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d2b270 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faff0012d00 tx=0x7faff0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aa2e6044ee25d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7faffe500640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fafe40a2e40 0x7fafe40a5300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faff0010c40 tx=0x7faff000c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7fafe3fff640 1 -- 172.21.15.31:0/2444763522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0024070 con 0x55a8a1d2ae50 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7fafe3fff640 1 -- 172.21.15.31:0/2444763522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faff0009d20 con 0x55a8a1d2ae50 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.089+0000 7fafe3fff640 1 -- 172.21.15.31:0/2444763522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0005240 con 0x55a8a1d2ae50 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 -- 172.21.15.31:0/2444763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 msgr2=0x55a8a1d2b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d2b270 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faff0012d00 tx=0x7faff0010740 comp rx=0 tx=0).stop 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 -- 172.21.15.31:0/2444763522 shutdown_connections 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d32e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2444763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d2b270 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 -- 172.21.15.31:0/2444763522 >> 172.21.15.31:0/2444763522 conn(0x55a8a1d264c0 msgr2=0x55a8a1d268c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 -- 172.21.15.31:0/2444763522 shutdown_connections 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.090+0000 7fb002c6a740 1 -- 172.21.15.31:0/2444763522 wait complete. 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 -- start start 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 Processor -- start 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 0x55a8a1d35820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a8a1d33900 con 0x55a8a1d2b840 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a8a1c4ab50 con 0x55a8a1d309b0 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a8a1d33a80 con 0x55a8a1d2ae50 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7fb002c6a740 10 client.0 _ll_register_callbacks cb 0x55a8a1c85ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7faffd4fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 0x55a8a1d35820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7faffd4fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 0x55a8a1d35820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41086/0 (socket says 172.21.15.31:41086) 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7faffd4fe640 1 -- 172.21.15.31:0/2788192042 learned_addr learned my addr 172.21.15.31:0/2788192042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7faffe500640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.091+0000 7faffdcff640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffd4fe640 1 -- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2ae50 msgr2=0x55a8a1d352e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffd4fe640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d352e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffd4fe640 1 -- 172.21.15.31:0/2788192042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 msgr2=0x55a8a1d3c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffd4fe640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffd4fe640 1 -- 172.21.15.31:0/2788192042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a1a6f620 con 0x55a8a1d2b840 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffe500640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7faffd4fe640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 0x55a8a1d35820 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fafe80314e0 tx=0x7fafe80bb7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cb7d028fcdd2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fafe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fafe17fa640 1 -- 172.21.15.31:0/2788192042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8035050 con 0x55a8a1d2b840 2024-04-25T00:39:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fafe17fa640 1 -- 172.21.15.31:0/2788192042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fafe803b460 con 0x55a8a1d2b840 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8a1c9c1f0 con 0x55a8a1d2b840 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fafe17fa640 1 -- 172.21.15.31:0/2788192042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafe8049040 con 0x55a8a1d2b840 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fb002c6a740 10 client.16125 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8a1d3fbd0 con 0x55a8a1d2b840 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.092+0000 7fafe17fa640 1 -- 172.21.15.31:0/2788192042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a8a1c9c1f0 con 0x55a8a1d2b840 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fafe17fa640 1 -- 172.21.15.31:0/2788192042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fafe80bc0b0 con 0x55a8a1d2b840 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fb002c6a740 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fb002c6a740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fb002c6a740 20 client.16125 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1d413d0 0x55a8a1d43810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fb002c6a740 4 client.16125 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8a1c96540 con 0x55a8a1d413d0 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7faffdcff640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1d413d0 0x55a8a1d43810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7faffdcff640 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1d413d0 0x55a8a1d43810 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a95647081b0989 server_cookie=8bd62983032863d9 in_seq=0 out_seq=0 2024-04-25T00:39:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.093+0000 7fafe17fa640 10 client.16125 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.094+0000 7fafe17fa640 1 -- 172.21.15.31:0/2788192042 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8a1c96540 con 0x55a8a1d413d0 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.094+0000 7fafe17fa640 10 client.16125 handle_command_reply: tid=0 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.094+0000 7fb002c6a740 1 client.16125 shutdown 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.094+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 msgr2=0x55a8a1d35820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.094+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 0x55a8a1d35820 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fafe80314e0 tx=0x7fafe80bb7b0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 shutdown_connections 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1d413d0 0x55a8a1d43810 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2b840 0x55a8a1d35820 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fafe17fa640 0 client.16125 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 --2- 172.21.15.31:0/2788192042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 >> 172.21.15.31:0/2788192042 conn(0x55a8a1d264c0 msgr2=0x55a8a1d255c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 shutdown_connections 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 1 -- 172.21.15.31:0/2788192042 wait complete. 2024-04-25T00:39:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.095+0000 7fb002c6a740 20 client.16125 trim_cache size 0 max 16384 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 Processor -- start 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 -- start start 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faff81a8b40 con 0x55a8a1d2ae50 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faff81aa680 con 0x55a8a1d309b0 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7fb002c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faff81a9210 con 0x55a8a1d2b840 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d24d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41100/0 (socket says 172.21.15.31:41100) 2024-04-25T00:39:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 -- 172.21.15.31:0/3337729925 learned_addr learned my addr 172.21.15.31:0/3337729925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffe500640 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffd4fe640 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 -- 172.21.15.31:0/3337729925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 msgr2=0x55a8a1d3c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 -- 172.21.15.31:0/3337729925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 msgr2=0x55a8a1d25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffdcff640 1 -- 172.21.15.31:0/3337729925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a1d43e90 con 0x55a8a1d2ae50 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.110+0000 7faffe500640 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7faffdcff640 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d24d30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faff400dc10 tx=0x7faff4011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb560bfb0f3ac6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fafe37fe640 1 -- 172.21.15.31:0/3337729925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4021030 con 0x55a8a1d2ae50 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fafe37fe640 1 -- 172.21.15.31:0/3337729925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faff4005020 con 0x55a8a1d2ae50 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fafe37fe640 1 -- 172.21.15.31:0/3337729925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4004a90 con 0x55a8a1d2ae50 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 -- 172.21.15.31:0/3337729925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 msgr2=0x55a8a1d24d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d24d30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faff400dc10 tx=0x7faff4011980 comp rx=0 tx=0).stop 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 -- 172.21.15.31:0/3337729925 shutdown_connections 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d309b0 0x55a8a1d25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 --2- 172.21.15.31:0/3337729925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d24d30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 -- 172.21.15.31:0/3337729925 >> 172.21.15.31:0/3337729925 conn(0x55a8a1d264c0 msgr2=0x55a8a1d2cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 -- 172.21.15.31:0/3337729925 shutdown_connections 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.111+0000 7fb002c6a740 1 -- 172.21.15.31:0/3337729925 wait complete. 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 -- start start 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 Processor -- start 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d39f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faff81a9fb0 con 0x55a8a1d309b0 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faff81a98e0 con 0x55a8a1d2ae50 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faff81a6b20 con 0x55a8a1d2b840 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7fb002c6a740 10 client.0 _ll_register_callbacks cb 0x7fafe80b8190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7faffe500640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7faffe500640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41110/0 (socket says 172.21.15.31:41110) 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7faffe500640 1 -- 172.21.15.31:0/97816292 learned_addr learned my addr 172.21.15.31:0/97816292 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7faffdcff640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d39f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.112+0000 7faffd4fe640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffe500640 1 -- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 msgr2=0x55a8a1d3a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffe500640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffe500640 1 -- 172.21.15.31:0/97816292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d2ae50 msgr2=0x55a8a1d39f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffe500640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d39f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffe500640 1 -- 172.21.15.31:0/97816292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a1d3ba40 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffdcff640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d39f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7faffe500640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3b450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faff0009a80 tx=0x7faff000cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b714d30f14087852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fafe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fafe17fa640 1 -- 172.21.15.31:0/97816292 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0024040 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fafe17fa640 1 -- 172.21.15.31:0/97816292 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7faff00057a0 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff4005020 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fafe17fa640 1 -- 172.21.15.31:0/97816292 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faff0012520 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fb002c6a740 10 client.16131 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8a1da7990 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.113+0000 7fafe17fa640 1 -- 172.21.15.31:0/97816292 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faff4005020 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fafe17fa640 1 -- 172.21.15.31:0/97816292 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faff003a070 con 0x55a8a1d309b0 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fb002c6a740 10 client.16131 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fb002c6a740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fb002c6a740 20 client.16131 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fb002c6a740 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1da8130 0x55a8a1daa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fb002c6a740 4 client.16131 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8a1c148e0 con 0x55a8a1da8130 2024-04-25T00:39:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.114+0000 7faffdcff640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1da8130 0x55a8a1daa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.115+0000 7fafe17fa640 10 client.16131 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.115+0000 7faffdcff640 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1da8130 0x55a8a1daa550 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737cf639542057d4 server_cookie=b53e5926cba78160 in_seq=0 out_seq=0 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fafe17fa640 1 -- 172.21.15.31:0/97816292 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8a1c148e0 con 0x55a8a1da8130 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fafe17fa640 10 client.16131 handle_command_reply: tid=0 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 client.16131 shutdown 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 msgr2=0x55a8a1d3b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3b450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faff0009a80 tx=0x7faff000cdf0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 shutdown_connections 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55a8a1da8130 0x55a8a1daa550 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8a1d309b0 0x55a8a1d3b450 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8a1d2b840 0x55a8a1d3a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 --2- 172.21.15.31:0/97816292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a8a1d2ae50 0x55a8a1d39f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fafe17fa640 0 client.16131 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 >> 172.21.15.31:0/97816292 conn(0x55a8a1d264c0 msgr2=0x55a8a1d29240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 shutdown_connections 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 1 -- 172.21.15.31:0/97816292 wait complete. 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.131+0000 7fb002c6a740 20 client.16131 trim_cache size 0 max 16384 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:39:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fafe40a2e40 msgr2=0x7fafe40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fafe40a2e40 0x7fafe40a5300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faff0010c40 tx=0x7faff000c000 comp rx=0 tx=0).stop 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 msgr2=0x7faff8197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8197ec0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fafe8004770 tx=0x7fafe80042f0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 shutdown_connections 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff8102250 0x7faff819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fafe40a2e40 0x7fafe40a5300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff8101890 0x7faff8197ec0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 --2- 172.21.15.31:0/3258264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faff81006a0 0x7faff8197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 >> 172.21.15.31:0/3258264776 conn(0x7faff80fbc10 msgr2=0x7faff8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 shutdown_connections 2024-04-25T00:39:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:42.132+0000 7faffff63640 1 -- 172.21.15.31:0/3258264776 wait complete. 2024-04-25T00:39:43.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:43 smithi031 ceph-mon[29167]: pgmap v635: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:43.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:43 smithi169 ceph-mon[33112]: pgmap v635: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:43.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:43 smithi040 ceph-mon[32945]: pgmap v635: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:39:45.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:45 smithi031 ceph-mon[29167]: pgmap v636: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 23 op/s 2024-04-25T00:39:45.641 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:45 smithi169 ceph-mon[33112]: pgmap v636: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 23 op/s 2024-04-25T00:39:45.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:45 smithi040 ceph-mon[32945]: pgmap v636: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 23 op/s 2024-04-25T00:39:47.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:47 smithi031 ceph-mon[29167]: pgmap v637: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:39:47.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:47 smithi169 ceph-mon[33112]: pgmap v637: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:39:47.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:47 smithi040 ceph-mon[32945]: pgmap v637: 97 pgs: 97 active+clean; 2.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:39:49.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:49 smithi031 ceph-mon[29167]: pgmap v638: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:49.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:49 smithi169 ceph-mon[33112]: pgmap v638: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:49.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:49 smithi040 ceph-mon[32945]: pgmap v638: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:39:51.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:51 smithi031 ceph-mon[29167]: pgmap v639: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:39:51.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:51 smithi169 ceph-mon[33112]: pgmap v639: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:39:51.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:51 smithi040 ceph-mon[32945]: pgmap v639: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:39:53.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:53 smithi031 ceph-mon[29167]: pgmap v640: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:53.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:53 smithi169 ceph-mon[33112]: pgmap v640: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:53.644 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:53 smithi040 ceph-mon[32945]: pgmap v640: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:39:54.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:39:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:39:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:39:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:39:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:39:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:39:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:39:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:39:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:39:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:39:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:39:54.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 -- 172.21.15.31:0/1241209753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0100040 msgr2=0x7febe0100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 --2- 172.21.15.31:0/1241209753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0100040 0x7febe0100460 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc02a410 comp rx=0 tx=0).stop 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 -- 172.21.15.31:0/1241209753 shutdown_connections 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 --2- 172.21.15.31:0/1241209753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe0101340 0x7febe01017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 --2- 172.21.15.31:0/1241209753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe01009a0 0x7febe0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 --2- 172.21.15.31:0/1241209753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0100040 0x7febe0100460 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 -- 172.21.15.31:0/1241209753 >> 172.21.15.31:0/1241209753 conn(0x7febe00fbc40 msgr2=0x7febe00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 -- 172.21.15.31:0/1241209753 shutdown_connections 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.230+0000 7febe7a95640 1 -- 172.21.15.31:0/1241209753 wait complete. 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 Processor -- start 2024-04-25T00:39:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 -- start start 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe0100040 0x7febe0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 0x7febe019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe0101340 0x7febe01a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7febe010f4c0 con 0x7febe01009a0 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febe010f340 con 0x7febe0101340 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe7a95640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febe010f640 con 0x7febe0100040 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.231+0000 7febe5030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 0x7febe019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 0x7febe019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35276/0 (socket says 172.21.15.31:35276) 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 -- 172.21.15.31:0/1607838410 learned_addr learned my addr 172.21.15.31:0/1607838410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5831640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe0100040 0x7febe0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe6032640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe0101340 0x7febe01a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 -- 172.21.15.31:0/1607838410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe0100040 msgr2=0x7febe0199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe0100040 0x7febe0199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 -- 172.21.15.31:0/1607838410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe0101340 msgr2=0x7febe01a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe0101340 0x7febe01a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 -- 172.21.15.31:0/1607838410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01a1730 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe5030640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 0x7febe019a050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7febd000e9f0 tx=0x7febd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febd6ffd640 1 -- 172.21.15.31:0/1607838410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000cd80 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01a1a20 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febd6ffd640 1 -- 172.21.15.31:0/1607838410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd00108e0 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febd6ffd640 1 -- 172.21.15.31:0/1607838410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0018920 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.232+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01a1fc0 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.235+0000 7febd6ffd640 1 -- 172.21.15.31:0/1607838410 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7febd0010a90 con 0x7febe01009a0 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.235+0000 7febea79c740 1 Processor -- start 2024-04-25T00:39:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.235+0000 7febea79c740 1 -- start start 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febea79c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febea79c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d43340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febea79c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d45e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febea79c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5646b5ca1cb0 con 0x5646b5d3e840 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febea79c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5646b5c4a5b0 con 0x5646b5d3de50 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febea79c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646b5c9dc70 con 0x5646b5d439b0 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febd6ffd640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7febcc0a2c30 0x7febcc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d43340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d43340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35290/0 (socket says 172.21.15.31:35290) 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 -- 172.21.15.31:0/108139774 learned_addr learned my addr 172.21.15.31:0/108139774 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe6032640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7febcc0a2c30 0x7febcc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febd6ffd640 1 -- 172.21.15.31:0/1607838410 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7febd00cf340 con 0x7febe01009a0 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5030640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d45e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe6032640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 -- 172.21.15.31:0/108139774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 msgr2=0x5646b5d45e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d45e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 -- 172.21.15.31:0/108139774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 msgr2=0x5646b5d3e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5831640 1 -- 172.21.15.31:0/108139774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b59b6210 con 0x5646b5d3e840 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe6032640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.236+0000 7febe5030640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d45e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febe5831640 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d43340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58f14fe331e1b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febe6032640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7febcc0a2c30 0x7febcc0a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5646b5d3f0a0 tx=0x7febd8009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febcbfff640 1 -- 172.21.15.31:0/108139774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc035030 con 0x5646b5d3e840 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febcbfff640 1 -- 172.21.15.31:0/108139774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febdc040070 con 0x5646b5d3e840 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febcbfff640 1 -- 172.21.15.31:0/108139774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc003a10 con 0x5646b5d3e840 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febea79c740 1 -- 172.21.15.31:0/108139774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 msgr2=0x5646b5d43340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.237+0000 7febea79c740 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d43340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc005c00 comp rx=0 tx=0).stop 2024-04-25T00:39:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 -- 172.21.15.31:0/108139774 shutdown_connections 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d45e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d43340 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 --2- 172.21.15.31:0/108139774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 -- 172.21.15.31:0/108139774 >> 172.21.15.31:0/108139774 conn(0x5646b5d394c0 msgr2=0x5646b5d398c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 -- 172.21.15.31:0/108139774 shutdown_connections 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 -- 172.21.15.31:0/108139774 wait complete. 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 -- start start 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.238+0000 7febea79c740 1 Processor -- start 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 0x5646b5d48270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3e840 0x5646b5d487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5646b5d46900 con 0x5646b5d3de50 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5646b5c5db50 con 0x5646b5d3e840 2024-04-25T00:39:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646b5d46a80 con 0x5646b5d439b0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febea79c740 10 client.0 _ll_register_callbacks cb 0x5646b5c98ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 0x5646b5d48270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 0x5646b5d48270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35296/0 (socket says 172.21.15.31:35296) 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 -- 172.21.15.31:0/2666608065 learned_addr learned my addr 172.21.15.31:0/2666608065 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe6032640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5030640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3e840 0x5646b5d487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 -- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 msgr2=0x5646b5d4f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 -- 172.21.15.31:0/2666608065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3e840 msgr2=0x5646b5d487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3e840 0x5646b5d487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 -- 172.21.15.31:0/2666608065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b5a82620 con 0x5646b5d3de50 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febe5831640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 0x5646b5d48270 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7febdc02a930 tx=0x7febdc031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538a4622e7dd8b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.239+0000 7febc97fa640 1 -- 172.21.15.31:0/2666608065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc051060 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febc97fa640 1 -- 172.21.15.31:0/2666608065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7febdc03c850 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febdc040070 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febc97fa640 1 -- 172.21.15.31:0/2666608065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7febdc035070 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 10 client.16143 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646b5d52ad0 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febc97fa640 1 -- 172.21.15.31:0/2666608065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febdc040070 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febc97fa640 1 -- 172.21.15.31:0/2666608065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7febdc003cc0 con 0x5646b5d3de50 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 20 client.16143 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5d53e20 0x5646b5d56210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 4 client.16143 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646b5caf1f0 con 0x5646b5d53e20 2024-04-25T00:39:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.240+0000 7febe5030640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5d53e20 0x5646b5d56210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.241+0000 7febe5030640 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5d53e20 0x5646b5d56210 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d701870bf1f0d5 server_cookie=6c725033581740df in_seq=0 out_seq=0 2024-04-25T00:39:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.241+0000 7febc97fa640 10 client.16143 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febc97fa640 1 -- 172.21.15.31:0/2666608065 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646b5caf1f0 con 0x5646b5d53e20 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febc97fa640 10 client.16143 handle_command_reply: tid=0 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 client.16143 shutdown 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 msgr2=0x5646b5d48270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 0x5646b5d48270 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7febdc02a930 tx=0x7febdc031830 comp rx=0 tx=0).stop 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 shutdown_connections 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5d53e20 0x5646b5d56210 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3e840 0x5646b5d487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febc97fa640 0 client.16143 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 --2- 172.21.15.31:0/2666608065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3de50 0x5646b5d48270 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 >> 172.21.15.31:0/2666608065 conn(0x5646b5d394c0 msgr2=0x5646b5d385c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 shutdown_connections 2024-04-25T00:39:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 1 -- 172.21.15.31:0/2666608065 wait complete. 2024-04-25T00:39:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.242+0000 7febea79c740 20 client.16143 trim_cache size 0 max 16384 2024-04-25T00:39:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.257+0000 7febea79c740 1 Processor -- start 2024-04-25T00:39:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.257+0000 7febea79c740 1 -- start start 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febea79c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febea79c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febea79c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d38440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febea79c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7febe01aad60 con 0x5646b5d3e840 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febea79c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ac8a0 con 0x5646b5d3de50 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febea79c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ab430 con 0x5646b5d439b0 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35298/0 (socket says 172.21.15.31:35298) 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 -- 172.21.15.31:0/1312087741 learned_addr learned my addr 172.21.15.31:0/1312087741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe6032640 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d38440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5831640 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 -- 172.21.15.31:0/1312087741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 msgr2=0x5646b5d38440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d38440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 -- 172.21.15.31:0/1312087741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 msgr2=0x5646b5d3cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 -- 172.21.15.31:0/1312087741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b5d567f0 con 0x5646b5d3e840 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5831640 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febe5030640 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4f7b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7febd0009250 tx=0x7febd00ccce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71ac3321fa2d20e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febcb7fe640 1 -- 172.21.15.31:0/1312087741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00b9070 con 0x5646b5d3e840 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febcb7fe640 1 -- 172.21.15.31:0/1312087741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd0099cc0 con 0x5646b5d3e840 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.258+0000 7febcb7fe640 1 -- 172.21.15.31:0/1312087741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd009b360 con 0x5646b5d3e840 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 -- 172.21.15.31:0/1312087741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 msgr2=0x5646b5d4f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4f7b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7febd0009250 tx=0x7febd00ccce0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 -- 172.21.15.31:0/1312087741 shutdown_connections 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d38440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4f7b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 --2- 172.21.15.31:0/1312087741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d3cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 -- 172.21.15.31:0/1312087741 >> 172.21.15.31:0/1312087741 conn(0x5646b5d394c0 msgr2=0x5646b5d37d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 -- 172.21.15.31:0/1312087741 shutdown_connections 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 -- 172.21.15.31:0/1312087741 wait complete. 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 -- start start 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.259+0000 7febea79c740 1 Processor -- start 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d4ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ac1d0 con 0x5646b5d3e840 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febe01abb00 con 0x5646b5d3de50 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febe01a8d40 con 0x5646b5d439b0 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febea79c740 10 client.0 _ll_register_callbacks cb 0x7febdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35310/0 (socket says 172.21.15.31:35310) 2024-04-25T00:39:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 -- 172.21.15.31:0/1568532989 learned_addr learned my addr 172.21.15.31:0/1568532989 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe6032640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5831640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d4ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 -- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 msgr2=0x5646b5d4e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 -- 172.21.15.31:0/1568532989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 msgr2=0x5646b5d4ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d4ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.260+0000 7febe5030640 1 -- 172.21.15.31:0/1568532989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b5d4e9a0 con 0x5646b5d3e840 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febe5030640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4d3f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7febd009b920 tx=0x7febd0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11f19c23d2f0ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febc97fa640 1 -- 172.21.15.31:0/1568532989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00a8070 con 0x5646b5d3e840 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febc97fa640 1 -- 172.21.15.31:0/1568532989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7febd009b2e0 con 0x5646b5d3e840 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd0099cc0 con 0x5646b5d3e840 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febc97fa640 1 -- 172.21.15.31:0/1568532989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7febd00b9050 con 0x5646b5d3e840 2024-04-25T00:39:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 10 client.16149 fetch_fsmap learned FSMap version 26 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646b5dba2d0 con 0x5646b5d3e840 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febc97fa640 1 -- 172.21.15.31:0/1568532989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd0099cc0 con 0x5646b5d3e840 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febc97fa640 1 -- 172.21.15.31:0/1568532989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7febd00ab020 con 0x5646b5d3e840 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 20 client.16149 populate_metadata read hostname 'smithi031' 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5dbaba0 0x5646b5dbcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 4 client.16149 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.261+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646b5ca9540 con 0x5646b5dbaba0 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.262+0000 7febe5831640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5dbaba0 0x5646b5dbcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.262+0000 7febc97fa640 10 client.16149 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.262+0000 7febe5831640 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5dbaba0 0x5646b5dbcfc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f2f9bdc60a280d server_cookie=35802f440f3e2fd5 in_seq=0 out_seq=0 2024-04-25T00:39:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febc97fa640 1 -- 172.21.15.31:0/1568532989 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646b5ca9540 con 0x5646b5dbaba0 2024-04-25T00:39:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febc97fa640 10 client.16149 handle_command_reply: tid=0 2024-04-25T00:39:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 client.16149 shutdown 2024-04-25T00:39:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 msgr2=0x5646b5d4d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4d3f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7febd009b920 tx=0x7febd0009770 comp rx=0 tx=0).stop 2024-04-25T00:39:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 shutdown_connections 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5646b5dbaba0 0x5646b5dbcfc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646b5d439b0 0x5646b5d4e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646b5d3e840 0x5646b5d4d3f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 --2- 172.21.15.31:0/1568532989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5646b5d3de50 0x5646b5d4ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febc97fa640 0 client.16149 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 >> 172.21.15.31:0/1568532989 conn(0x5646b5d394c0 msgr2=0x5646b5d385c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 shutdown_connections 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.278+0000 7febea79c740 1 -- 172.21.15.31:0/1568532989 wait complete. 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febea79c740 20 client.16149 trim_cache size 0 max 16384 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:39:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:39:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:39:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:39:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7febcc0a2c30 msgr2=0x7febcc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7febcc0a2c30 0x7febcc0a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5646b5d3f0a0 tx=0x7febd8009be0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 msgr2=0x7febe019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 0x7febe019a050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7febd000e9f0 tx=0x7febd000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 shutdown_connections 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe0101340 0x7febe01a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7febcc0a2c30 0x7febcc0a50f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe01009a0 0x7febe019a050 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 --2- 172.21.15.31:0/1607838410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe0100040 0x7febe0199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 >> 172.21.15.31:0/1607838410 conn(0x7febe00fbc40 msgr2=0x7febe00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 shutdown_connections 2024-04-25T00:39:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:39:55.279+0000 7febe7a95640 1 -- 172.21.15.31:0/1607838410 wait complete. 2024-04-25T00:39:55.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:55 smithi169 ceph-mon[33112]: pgmap v641: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:55.646 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:55 smithi040 ceph-mon[32945]: pgmap v641: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:55.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:55 smithi031 ceph-mon[29167]: pgmap v641: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:39:56.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:56 smithi169 ceph-mon[33112]: pgmap v642: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:39:56.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:56 smithi040 ceph-mon[32945]: pgmap v642: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:39:56.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:56 smithi031 ceph-mon[29167]: pgmap v642: 97 pgs: 97 active+clean; 2.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:39:58.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:39:58 smithi169 ceph-mon[33112]: pgmap v643: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-25T00:39:58.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:39:58 smithi040 ceph-mon[32945]: pgmap v643: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-25T00:39:59.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:39:58 smithi031 ceph-mon[29167]: pgmap v643: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-25T00:40:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:00 smithi169 ceph-mon[33112]: pgmap v644: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:00.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:00 smithi169 ceph-mon[33112]: overall HEALTH_OK 2024-04-25T00:40:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:00 smithi040 ceph-mon[32945]: pgmap v644: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:00 smithi040 ceph-mon[32945]: overall HEALTH_OK 2024-04-25T00:40:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:00 smithi031 ceph-mon[29167]: pgmap v644: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:00 smithi031 ceph-mon[29167]: overall HEALTH_OK 2024-04-25T00:40:02.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:02 smithi169 ceph-mon[33112]: pgmap v645: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:02 smithi040 ceph-mon[32945]: pgmap v645: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:03.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:02 smithi031 ceph-mon[29167]: pgmap v645: 97 pgs: 97 active+clean; 2.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:04 smithi169 ceph-mon[33112]: pgmap v646: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:04 smithi040 ceph-mon[32945]: pgmap v646: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:05.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:04 smithi031 ceph-mon[29167]: pgmap v646: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:06.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:06 smithi031 ceph-mon[29167]: pgmap v647: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:06 smithi169 ceph-mon[33112]: pgmap v647: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:06 smithi040 ceph-mon[32945]: pgmap v647: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 -- 172.21.15.31:0/2065260645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 msgr2=0x7f85ec100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 --2- 172.21.15.31:0/2065260645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec100460 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009900 tx=0x7f85e802a410 comp rx=0 tx=0).stop 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 -- 172.21.15.31:0/2065260645 shutdown_connections 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 --2- 172.21.15.31:0/2065260645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85ec101340 0x7f85ec1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 --2- 172.21.15.31:0/2065260645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ec1009a0 0x7f85ec100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 --2- 172.21.15.31:0/2065260645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec100460 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 -- 172.21.15.31:0/2065260645 >> 172.21.15.31:0/2065260645 conn(0x7f85ec0fbc40 msgr2=0x7f85ec0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 -- 172.21.15.31:0/2065260645 shutdown_connections 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 -- 172.21.15.31:0/2065260645 wait complete. 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 Processor -- start 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.378+0000 7f85f3bed640 1 -- start start 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f3bed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f3bed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ec1009a0 0x7f85ec19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f3bed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85ec101340 0x7f85ec1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f3bed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec10f4c0 con 0x7f85ec100040 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f3bed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec10f340 con 0x7f85ec1009a0 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f3bed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec10f640 con 0x7f85ec101340 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50282/0 (socket says 172.21.15.31:50282) 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 -- 172.21.15.31:0/3753516472 learned_addr learned my addr 172.21.15.31:0/3753516472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f218a640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85ec101340 0x7f85ec1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1188640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ec1009a0 0x7f85ec19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 -- 172.21.15.31:0/3753516472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85ec101340 msgr2=0x7f85ec1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85ec101340 0x7f85ec1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 -- 172.21.15.31:0/3753516472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ec1009a0 msgr2=0x7f85ec19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ec1009a0 0x7f85ec19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 -- 172.21.15.31:0/3753516472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec1a1700 con 0x7f85ec100040 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85f1989640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec199ae0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f85e8004770 tx=0x7f85e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.379+0000 7f85e2ffd640 1 -- 172.21.15.31:0/3753516472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e803f030 con 0x7f85ec100040 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.380+0000 7f85e2ffd640 1 -- 172.21.15.31:0/3753516472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85e8003cb0 con 0x7f85ec100040 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.380+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec1a1990 con 0x7f85ec100040 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.380+0000 7f85e2ffd640 1 -- 172.21.15.31:0/3753516472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8035070 con 0x7f85ec100040 2024-04-25T00:40:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.380+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a1e80 con 0x7f85ec100040 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.382+0000 7f85e2ffd640 1 -- 172.21.15.31:0/3753516472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85e8042030 con 0x7f85ec100040 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.382+0000 7f85f69cf740 1 Processor -- start 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 -- start start 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80927270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b8092c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b8092ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556b8088acb0 con 0x556b8092c9b0 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556b808335b0 con 0x556b80926e50 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f69cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556b80886c70 con 0x556b80927840 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85e2ffd640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85d80a2c30 0x7f85d80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f1188640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b8092ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f1188640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b8092ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50296/0 (socket says 172.21.15.31:50296) 2024-04-25T00:40:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f1188640 1 -- 172.21.15.31:0/2373557513 learned_addr learned my addr 172.21.15.31:0/2373557513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f218a640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85d80a2c30 0x7f85d80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85e2ffd640 1 -- 172.21.15.31:0/3753516472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f85e80efb00 con 0x7f85ec100040 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f218a640 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80927270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.383+0000 7f85f1989640 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b8092c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f1188640 1 -- 172.21.15.31:0/2373557513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 msgr2=0x556b8092c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f1188640 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b8092c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f1188640 1 -- 172.21.15.31:0/2373557513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 msgr2=0x556b80927270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f1188640 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80927270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f1188640 1 -- 172.21.15.31:0/2373557513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8059f210 con 0x556b8092c9b0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f218a640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85d80a2c30 0x7f85d80a50f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85e4005fd0 tx=0x7f85e4009d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85f1188640 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b8092ee40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009b60 tx=0x7f85dc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701e9b26123f063c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85d7fff640 1 -- 172.21.15.31:0/2373557513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc016070 con 0x556b8092c9b0 2024-04-25T00:40:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85d7fff640 1 -- 172.21.15.31:0/2373557513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85dc009e50 con 0x556b8092c9b0 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.384+0000 7f85d7fff640 1 -- 172.21.15.31:0/2373557513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004e00 con 0x556b8092c9b0 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 -- 172.21.15.31:0/2373557513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 msgr2=0x556b8092ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b8092ee40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009b60 tx=0x7f85dc00e6c0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 -- 172.21.15.31:0/2373557513 shutdown_connections 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b8092ee40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b8092c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 --2- 172.21.15.31:0/2373557513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80927270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 -- 172.21.15.31:0/2373557513 >> 172.21.15.31:0/2373557513 conn(0x556b809224c0 msgr2=0x556b809228c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 -- 172.21.15.31:0/2373557513 shutdown_connections 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 -- 172.21.15.31:0/2373557513 wait complete. 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 -- start start 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.385+0000 7f85f69cf740 1 Processor -- start 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80926e50 0x556b80931210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 0x556b80931750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b8092c9b0 0x556b80938790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556b8092f960 con 0x556b80927840 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556b80846b50 con 0x556b8092c9b0 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556b8092fae0 con 0x556b80926e50 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f69cf740 10 client.0 _ll_register_callbacks cb 0x556b80881ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 0x556b80931750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 0x556b80931750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50300/0 (socket says 172.21.15.31:50300) 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 -- 172.21.15.31:0/1675750776 learned_addr learned my addr 172.21.15.31:0/1675750776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1989640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80926e50 0x556b80931210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f218a640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b8092c9b0 0x556b80938790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 -- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80926e50 msgr2=0x556b80931210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80926e50 0x556b80931210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 -- 172.21.15.31:0/1675750776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b8092c9b0 msgr2=0x556b80938790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b8092c9b0 0x556b80938790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 -- 172.21.15.31:0/1675750776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8066b620 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85f1188640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 0x556b80931750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f85dc000c00 tx=0x7f85dc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118967a6e1c2beea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.386+0000 7f85d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85d57fa640 1 -- 172.21.15.31:0/1675750776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc028060 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85d57fa640 1 -- 172.21.15.31:0/1675750776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f85dc009c50 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc009e50 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85d57fa640 1 -- 172.21.15.31:0/1675750776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85dc016040 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 10 client.16161 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b8093bb50 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85d57fa640 1 -- 172.21.15.31:0/1675750776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85dc009e50 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85d57fa640 1 -- 172.21.15.31:0/1675750776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f85dc00f040 con 0x556b80927840 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 10 client.16161 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 20 client.16161 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b8093cf50 0x556b8093f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 4 client.16161 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.387+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b808981f0 con 0x556b8093cf50 2024-04-25T00:40:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.388+0000 7f85f1989640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b8093cf50 0x556b8093f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.388+0000 7f85f1989640 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b8093cf50 0x556b8093f340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58605e1adc04660e server_cookie=c2a56732e7a245f in_seq=0 out_seq=0 2024-04-25T00:40:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.388+0000 7f85d57fa640 10 client.16161 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85d57fa640 1 -- 172.21.15.31:0/1675750776 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b808981f0 con 0x556b8093cf50 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85d57fa640 10 client.16161 handle_command_reply: tid=0 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 client.16161 shutdown 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 msgr2=0x556b80931750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 0x556b80931750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f85dc000c00 tx=0x7f85dc011b40 comp rx=0 tx=0).stop 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 shutdown_connections 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b8093cf50 0x556b8093f340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b8092c9b0 0x556b80938790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80927840 0x556b80931750 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1675750776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80926e50 0x556b80931210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85d57fa640 0 client.16161 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 >> 172.21.15.31:0/1675750776 conn(0x556b809224c0 msgr2=0x556b809215c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 shutdown_connections 2024-04-25T00:40:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 1 -- 172.21.15.31:0/1675750776 wait complete. 2024-04-25T00:40:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.389+0000 7f85f69cf740 20 client.16161 trim_cache size 0 max 16384 2024-04-25T00:40:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 Processor -- start 2024-04-25T00:40:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 -- start start 2024-04-25T00:40:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 0x556b809280d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80927840 0x556b80938750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b8092c9b0 0x556b80920d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1aace0 con 0x556b80926e50 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ac820 con 0x556b80927840 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.404+0000 7f85f69cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ab3b0 con 0x556b8092c9b0 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 0x556b809280d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 0x556b809280d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50316/0 (socket says 172.21.15.31:50316) 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 -- 172.21.15.31:0/1468829689 learned_addr learned my addr 172.21.15.31:0/1468829689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f218a640 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b8092c9b0 0x556b80920d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1188640 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80927840 0x556b80938750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 -- 172.21.15.31:0/1468829689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b8092c9b0 msgr2=0x556b80920d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b8092c9b0 0x556b80920d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 -- 172.21.15.31:0/1468829689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80927840 msgr2=0x556b80938750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80927840 0x556b80938750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 -- 172.21.15.31:0/1468829689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8092bec0 con 0x556b80926e50 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85f1989640 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 0x556b809280d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f85e80ed5d0 tx=0x7f85e80ed4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7608ca5ea18f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85d77fe640 1 -- 172.21.15.31:0/1468829689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8049040 con 0x556b80926e50 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85d77fe640 1 -- 172.21.15.31:0/1468829689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85e80c06e0 con 0x556b80926e50 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.405+0000 7f85d77fe640 1 -- 172.21.15.31:0/1468829689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e803bc90 con 0x556b80926e50 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 -- 172.21.15.31:0/1468829689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 msgr2=0x556b809280d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 0x556b809280d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f85e80ed5d0 tx=0x7f85e80ed4a0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 -- 172.21.15.31:0/1468829689 shutdown_connections 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b8092c9b0 0x556b80920d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80927840 0x556b80938750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 --2- 172.21.15.31:0/1468829689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b80926e50 0x556b809280d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 -- 172.21.15.31:0/1468829689 >> 172.21.15.31:0/1468829689 conn(0x556b809224c0 msgr2=0x556b809228a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 -- 172.21.15.31:0/1468829689 shutdown_connections 2024-04-25T00:40:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 -- 172.21.15.31:0/1468829689 wait complete. 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 -- start start 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.406+0000 7f85f69cf740 1 Processor -- start 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80935d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b809362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b809372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ac150 con 0x556b8092c9b0 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1aba80 con 0x556b80926e50 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1a8cc0 con 0x556b80927840 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f69cf740 10 client.0 _ll_register_callbacks cb 0x7f85e4012650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f218a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b809372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b809362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f218a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b809372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50322/0 (socket says 172.21.15.31:50322) 2024-04-25T00:40:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f218a640 1 -- 172.21.15.31:0/3595179041 learned_addr learned my addr 172.21.15.31:0/3595179041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1989640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80935d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 -- 172.21.15.31:0/3595179041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 msgr2=0x556b80935d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80935d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 -- 172.21.15.31:0/3595179041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 msgr2=0x556b809372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b809372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 -- 172.21.15.31:0/3595179041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b809378d0 con 0x556b80927840 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f218a640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b809372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85f1188640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b809362d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00ebc0 tx=0x7f85dc0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250262faeb59c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.407+0000 7f85d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85d57fa640 1 -- 172.21.15.31:0/3595179041 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc028040 con 0x556b80927840 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e80c06e0 con 0x556b80927840 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85d57fa640 1 -- 172.21.15.31:0/3595179041 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f85dc01a7f0 con 0x556b80927840 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85d57fa640 1 -- 172.21.15.31:0/3595179041 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85dc009e60 con 0x556b80927840 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 10 client.25750 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b809a3430 con 0x556b80927840 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85d57fa640 1 -- 172.21.15.31:0/3595179041 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85e80c06e0 con 0x556b80927840 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85d57fa640 1 -- 172.21.15.31:0/3595179041 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f85dc02e6a0 con 0x556b80927840 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 10 client.25750 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 20 client.25750 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b809a3c50 0x556b809a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 4 client.25750 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b80892540 con 0x556b809a3c50 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.408+0000 7f85f1989640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b809a3c50 0x556b809a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.409+0000 7f85f1989640 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b809a3c50 0x556b809a6070 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e010a8294c742d server_cookie=27ba97b1f7909b81 in_seq=0 out_seq=0 2024-04-25T00:40:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.409+0000 7f85d57fa640 10 client.25750 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85d57fa640 1 -- 172.21.15.31:0/3595179041 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b80892540 con 0x556b809a3c50 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85d57fa640 10 client.25750 handle_command_reply: tid=0 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 client.25750 shutdown 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 msgr2=0x556b809362d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b809362d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00ebc0 tx=0x7f85dc0058a0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 shutdown_connections 2024-04-25T00:40:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x556b809a3c50 0x556b809a6070 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b8092c9b0 0x556b809372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85d57fa640 0 client.25750 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b80927840 0x556b809362d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 --2- 172.21.15.31:0/3595179041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b80926e50 0x556b80935d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 >> 172.21.15.31:0/3595179041 conn(0x556b809224c0 msgr2=0x556b809280d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 shutdown_connections 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.425+0000 7f85f69cf740 1 -- 172.21.15.31:0/3595179041 wait complete. 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f69cf740 20 client.25750 trim_cache size 0 max 16384 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:40:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:40:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:40:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:40:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85d80a2c30 msgr2=0x7f85d80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85d80a2c30 0x7f85d80a50f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85e4005fd0 tx=0x7f85e4009d10 comp rx=0 tx=0).stop 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 msgr2=0x7f85ec199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec199ae0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f85e8004770 tx=0x7f85e80042f0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 shutdown_connections 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85ec101340 0x7f85ec1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ec1009a0 0x7f85ec19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85d80a2c30 0x7f85d80a50f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 --2- 172.21.15.31:0/3753516472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85ec100040 0x7f85ec199ae0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 >> 172.21.15.31:0/3753516472 conn(0x7f85ec0fbc40 msgr2=0x7f85ec0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 shutdown_connections 2024-04-25T00:40:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:08.426+0000 7f85f3bed640 1 -- 172.21.15.31:0/3753516472 wait complete. 2024-04-25T00:40:08.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:08 smithi169 ceph-mon[33112]: pgmap v648: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:08.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:08 smithi040 ceph-mon[32945]: pgmap v648: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:08 smithi031 ceph-mon[29167]: pgmap v648: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:10.538 INFO:tasks.workunit.client.0.smithi031.stdout:437/600: open fd = 3 2024-04-25T00:40:10.538 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:438/600: open fd = 3 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:439/600: open fd = 3 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:440/600: open fd = 3 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:441/600: open fd = 3 2024-04-25T00:40:10.539 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:442/600: open fd = 3 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:443/600: open fd = 3 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:444/600: open fd = 3 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:445/600: open fd = 3 2024-04-25T00:40:10.540 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:446/600: open fd = 3 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:447/600: open fd = 3 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:448/600: open fd = 3 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:449/600: open fd = 3 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.541 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:450/600: open fd = 3 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:451/600: open fd = 3 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:452/600: open fd = 3 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:453/600: open fd = 3 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.542 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:454/600: open fd = 3 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:455/600: open fd = 3 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:456/600: open fd = 3 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:457/600: open fd = 3 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.543 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:458/600: open fd = 3 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:459/600: open fd = 3 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:460/600: open fd = 3 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:461/600: open fd = 3 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.544 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:462/600: open fd = 3 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:463/600: open fd = 3 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:464/600: open fd = 3 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:465/600: open fd = 3 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.545 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:466/600: open fd = 3 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:467/600: open fd = 3 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:468/600: open fd = 3 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:469/600: open fd = 3 2024-04-25T00:40:10.546 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:470/600: open fd = 3 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:471/600: open fd = 3 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:472/600: open fd = 3 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:473/600: open fd = 3 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.547 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:474/600: open fd = 3 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:475/600: open fd = 3 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:476/600: open fd = 3 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:477/600: open fd = 3 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.548 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:478/600: open fd = 3 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:479/600: open fd = 3 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:480/600: open fd = 3 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:481/600: open fd = 3 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.549 INFO:tasks.workunit.client.0.smithi031.stdout:482/600: open fd = 3 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:483/600: open fd = 3 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:484/600: open fd = 3 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:485/600: open fd = 3 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:486/600: open fd = 3 2024-04-25T00:40:10.550 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:487/600: open fd = 3 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:488/600: open fd = 3 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:489/600: open fd = 3 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:490/600: open fd = 3 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.551 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:491/600: open fd = 3 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:492/600: open fd = 3 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:493/600: open fd = 3 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:494/600: open fd = 3 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.552 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:495/600: open fd = 3 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:496/600: open fd = 3 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:497/600: open fd = 3 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:498/600: open fd = 3 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:40:10.553 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:40:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:10 smithi169 ceph-mon[33112]: pgmap v649: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:40:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:10 smithi040 ceph-mon[32945]: pgmap v649: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:40:11.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:10 smithi031 ceph-mon[29167]: pgmap v649: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T00:40:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:12 smithi169 ceph-mon[33112]: pgmap v650: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:40:12.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:12 smithi040 ceph-mon[32945]: pgmap v650: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:40:13.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:12 smithi031 ceph-mon[29167]: pgmap v650: 97 pgs: 97 active+clean; 2.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:40:14.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:14 smithi169 ceph-mon[33112]: pgmap v651: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:14.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:14 smithi040 ceph-mon[32945]: pgmap v651: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:15.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:14 smithi031 ceph-mon[29167]: pgmap v651: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:16 smithi169 ceph-mon[33112]: pgmap v652: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:16.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:16 smithi040 ceph-mon[32945]: pgmap v652: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:16 smithi031 ceph-mon[29167]: pgmap v652: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:18 smithi169 ceph-mon[33112]: pgmap v653: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:18.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:18 smithi040 ceph-mon[32945]: pgmap v653: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:18 smithi031 ceph-mon[29167]: pgmap v653: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:20.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:20 smithi169 ceph-mon[33112]: pgmap v654: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:20 smithi040 ceph-mon[32945]: pgmap v654: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:21.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:20 smithi031 ceph-mon[29167]: pgmap v654: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 -- 172.21.15.31:0/4060368057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c1006a0 msgr2=0x7fe72c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 --2- 172.21.15.31:0/4060368057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c1006a0 0x7fe72c100ac0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009900 tx=0x7fe71c02a410 comp rx=0 tx=0).stop 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 -- 172.21.15.31:0/4060368057 shutdown_connections 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 --2- 172.21.15.31:0/4060368057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe72c102250 0x7fe72c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 --2- 172.21.15.31:0/4060368057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 0x7fe72c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 --2- 172.21.15.31:0/4060368057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c1006a0 0x7fe72c100ac0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 -- 172.21.15.31:0/4060368057 >> 172.21.15.31:0/4060368057 conn(0x7fe72c0fbc10 msgr2=0x7fe72c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 -- 172.21.15.31:0/4060368057 shutdown_connections 2024-04-25T00:40:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 -- 172.21.15.31:0/4060368057 wait complete. 2024-04-25T00:40:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.525+0000 7fe7326ba640 1 Processor -- start 2024-04-25T00:40:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 -- start start 2024-04-25T00:40:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe72c1006a0 0x7fe72c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 0x7fe72c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 0x7fe72c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c10c1a0 con 0x7fe72c102250 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c10c020 con 0x7fe72c101890 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe7326ba640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c10c320 con 0x7fe72c1006a0 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 0x7fe72c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 0x7fe72c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50626/0 (socket says 172.21.15.31:50626) 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 -- 172.21.15.31:0/2460513596 learned_addr learned my addr 172.21.15.31:0/2460513596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe72bfff640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe72c1006a0 0x7fe72c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 0x7fe72c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 -- 172.21.15.31:0/2460513596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe72c1006a0 msgr2=0x7fe72c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe72c1006a0 0x7fe72c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 -- 172.21.15.31:0/2460513596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 msgr2=0x7fe72c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 0x7fe72c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe730c57640 1 -- 172.21.15.31:0/2460513596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c19f550 con 0x7fe72c102250 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.526+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 0x7fe72c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.527+0000 7fe730c57640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 0x7fe72c19eee0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe724009fd0 tx=0x7fe72400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.527+0000 7fe7297fa640 1 -- 172.21.15.31:0/2460513596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724017070 con 0x7fe72c102250 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.527+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c19f840 con 0x7fe72c102250 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.527+0000 7fe7297fa640 1 -- 172.21.15.31:0/2460513596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe724003e20 con 0x7fe72c102250 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.527+0000 7fe7297fa640 1 -- 172.21.15.31:0/2460513596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724010460 con 0x7fe72c102250 2024-04-25T00:40:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.527+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c19fd40 con 0x7fe72c102250 2024-04-25T00:40:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.529+0000 7fe7297fa640 1 -- 172.21.15.31:0/2460513596 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe724003970 con 0x7fe72c102250 2024-04-25T00:40:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.529+0000 7fe7353c6740 1 Processor -- start 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.529+0000 7fe7353c6740 1 -- start start 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7353c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f066703cb0 con 0x55f0667a4c50 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7353c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f0666ac5b0 con 0x55f0667a9dc0 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7353c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f0666ffc70 con 0x55f0667a4260 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7297fa640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe7140a2c30 0x7fe7140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667a9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50640/0 (socket says 172.21.15.31:50640) 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 -- 172.21.15.31:0/4014228007 learned_addr learned my addr 172.21.15.31:0/4014228007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe730c57640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe7140a2c30 0x7fe7140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe7297fa640 1 -- 172.21.15.31:0/2460513596 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe7240cef70 con 0x7fe72c102250 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 -- 172.21.15.31:0/4014228007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 msgr2=0x55f0667a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 -- 172.21.15.31:0/4014228007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 msgr2=0x55f0667ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.530+0000 7fe72bfff640 1 -- 172.21.15.31:0/4014228007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f066418210 con 0x55f0667a4c50 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe730c57640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe7140a2c30 0x7fe7140a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe72409b4a0 tx=0x7fe724066370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe72bfff640 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667a9750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009900 tx=0x7fe71c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2381fdf3c8b591b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe7127fc640 1 -- 172.21.15.31:0/4014228007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c03f070 con 0x55f0667a4c50 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe7127fc640 1 -- 172.21.15.31:0/4014228007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe71c031780 con 0x55f0667a4c50 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe7127fc640 1 -- 172.21.15.31:0/4014228007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c035070 con 0x55f0667a4c50 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe7353c6740 1 -- 172.21.15.31:0/4014228007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 msgr2=0x55f0667a9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.531+0000 7fe7353c6740 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667a9750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009900 tx=0x7fe71c032cc0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 -- 172.21.15.31:0/4014228007 shutdown_connections 2024-04-25T00:40:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667a9750 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 --2- 172.21.15.31:0/4014228007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 -- 172.21.15.31:0/4014228007 >> 172.21.15.31:0/4014228007 conn(0x55f06679b4c0 msgr2=0x55f06679b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 -- 172.21.15.31:0/4014228007 shutdown_connections 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 -- 172.21.15.31:0/4014228007 wait complete. 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 -- start start 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 Processor -- start 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 0x55f0667ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4c50 0x55f0667aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.532+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe7353c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f0667acd10 con 0x55f0667a4260 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe7353c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f0666bfb50 con 0x55f0667a9dc0 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe7353c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f0667ace90 con 0x55f0667a4c50 2024-04-25T00:40:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe7353c6740 10 client.0 _ll_register_callbacks cb 0x55f0666faca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 0x55f0667ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 0x55f0667ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50654/0 (socket says 172.21.15.31:50654) 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 -- 172.21.15.31:0/1226513404 learned_addr learned my addr 172.21.15.31:0/1226513404 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72bfff640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4c50 0x55f0667aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 -- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4c50 msgr2=0x55f0667aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4c50 0x55f0667aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 -- 172.21.15.31:0/1226513404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 msgr2=0x55f0667b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 -- 172.21.15.31:0/1226513404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0664e4620 con 0x55f0667a4260 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 0x55f0667ae690 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe72000fd30 tx=0x7fe7200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461039e90e5446d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe70b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe70b7fe640 1 -- 172.21.15.31:0/1226513404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720005020 con 0x55f0667a4260 2024-04-25T00:40:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe70b7fe640 1 -- 172.21.15.31:0/1226513404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe7200051c0 con 0x55f0667a4260 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.533+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71c031780 con 0x55f0667a4260 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe70b7fe640 1 -- 172.21.15.31:0/1226513404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe720026050 con 0x55f0667a4260 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 10 client.16179 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0667b9040 con 0x55f0667a4260 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe70b7fe640 1 -- 172.21.15.31:0/1226513404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe71c031780 con 0x55f0667a4260 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe70b7fe640 1 -- 172.21.15.31:0/1226513404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe720012450 con 0x55f0667a4260 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 10 client.16179 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 20 client.16179 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0667ba6c0 0x55f0667bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 4 client.16179 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0667111f0 con 0x55f0667ba6c0 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.534+0000 7fe72bfff640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0667ba6c0 0x55f0667bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.535+0000 7fe72bfff640 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0667ba6c0 0x55f0667bcb30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca5bf5838d7e15 server_cookie=fa10cee4e2becfd6 in_seq=0 out_seq=0 2024-04-25T00:40:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.535+0000 7fe70b7fe640 10 client.16179 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe70b7fe640 1 -- 172.21.15.31:0/1226513404 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0667111f0 con 0x55f0667ba6c0 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe70b7fe640 10 client.16179 handle_command_reply: tid=0 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 client.16179 shutdown 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 msgr2=0x55f0667ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 0x55f0667ae690 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe72000fd30 tx=0x7fe7200046c0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 shutdown_connections 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0667ba6c0 0x55f0667bcb30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4c50 0x55f0667aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 --2- 172.21.15.31:0/1226513404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4260 0x55f0667ae690 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe70b7fe640 0 client.16179 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 >> 172.21.15.31:0/1226513404 conn(0x55f06679b4c0 msgr2=0x55f06679a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 shutdown_connections 2024-04-25T00:40:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 1 -- 172.21.15.31:0/1226513404 wait complete. 2024-04-25T00:40:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.536+0000 7fe7353c6740 20 client.16179 trim_cache size 0 max 16384 2024-04-25T00:40:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 Processor -- start 2024-04-25T00:40:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 -- start start 2024-04-25T00:40:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a4260 0x55f06679e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a9dc0 0x55f06679e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1a68b0 con 0x55f0667a4c50 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1a83f0 con 0x55f0667a4260 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe7353c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1a6f80 con 0x55f0667a9dc0 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50658/0 (socket says 172.21.15.31:50658) 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 -- 172.21.15.31:0/584792071 learned_addr learned my addr 172.21.15.31:0/584792071 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a4260 0x55f06679e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a9dc0 0x55f06679e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 -- 172.21.15.31:0/584792071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a9dc0 msgr2=0x55f06679e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a9dc0 0x55f06679e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 -- 172.21.15.31:0/584792071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a4260 msgr2=0x55f06679e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a4260 0x55f06679e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72bfff640 1 -- 172.21.15.31:0/584792071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06679b060 con 0x55f0667a4c50 2024-04-25T00:40:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.551+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a4260 0x55f06679e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.552+0000 7fe72bfff640 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b5bd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe71c02a450 tx=0x7fe71c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad006d2b3a2dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.552+0000 7fe711ffb640 1 -- 172.21.15.31:0/584792071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c051070 con 0x55f0667a4c50 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.552+0000 7fe711ffb640 1 -- 172.21.15.31:0/584792071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe71c04cbc0 con 0x55f0667a4c50 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.552+0000 7fe711ffb640 1 -- 172.21.15.31:0/584792071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c03f040 con 0x55f0667a4c50 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.552+0000 7fe7353c6740 1 -- 172.21.15.31:0/584792071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 msgr2=0x55f0667b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.552+0000 7fe7353c6740 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b5bd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe71c02a450 tx=0x7fe71c005fd0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- 172.21.15.31:0/584792071 shutdown_connections 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a9dc0 0x55f06679e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b5bd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 --2- 172.21.15.31:0/584792071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a4260 0x55f06679e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- 172.21.15.31:0/584792071 >> 172.21.15.31:0/584792071 conn(0x55f06679b4c0 msgr2=0x55f0667ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- 172.21.15.31:0/584792071 shutdown_connections 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- 172.21.15.31:0/584792071 wait complete. 2024-04-25T00:40:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- start start 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 Processor -- start 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1a7d20 con 0x55f0667a4c50 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1a7650 con 0x55f0667a9dc0 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1a4890 con 0x55f0667a4260 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe7353c6740 10 client.0 _ll_register_callbacks cb 0x7fe72000f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50674/0 (socket says 172.21.15.31:50674) 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe72bfff640 1 -- 172.21.15.31:0/3536006289 learned_addr learned my addr 172.21.15.31:0/3536006289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.553+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72bfff640 1 -- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 msgr2=0x55f0667b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72bfff640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72bfff640 1 -- 172.21.15.31:0/3536006289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 msgr2=0x55f0667b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72bfff640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72bfff640 1 -- 172.21.15.31:0/3536006289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0667b4da0 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe72bfff640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b37a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe71c031580 tx=0x7fe71c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65782a5fd26cce6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe70b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe70b7fe640 1 -- 172.21.15.31:0/3536006289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c059060 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe70b7fe640 1 -- 172.21.15.31:0/3536006289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe71c04c4d0 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71c04cbc0 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe70b7fe640 1 -- 172.21.15.31:0/3536006289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe71c03f070 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe7353c6740 10 client.16185 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f066820cb0 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.554+0000 7fe70b7fe640 1 -- 172.21.15.31:0/3536006289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe71c04cbc0 con 0x55f0667a4c50 2024-04-25T00:40:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe70b7fe640 1 -- 172.21.15.31:0/3536006289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe71c03bb60 con 0x55f0667a4c50 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe7353c6740 10 client.16185 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe7353c6740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe7353c6740 20 client.16185 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe7353c6740 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0668210e0 0x55f066823500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe7353c6740 4 client.16185 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f06670b540 con 0x55f0668210e0 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0668210e0 0x55f066823500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe70b7fe640 10 client.16185 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.555+0000 7fe72b7fe640 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0668210e0 0x55f066823500 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd8aac3d605608d server_cookie=ea8f436c9cedc599 in_seq=0 out_seq=0 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.572+0000 7fe70b7fe640 1 -- 172.21.15.31:0/3536006289 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f06670b540 con 0x55f0668210e0 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.572+0000 7fe70b7fe640 10 client.16185 handle_command_reply: tid=0 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.572+0000 7fe7353c6740 1 client.16185 shutdown 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 msgr2=0x55f0667b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b37a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe71c031580 tx=0x7fe71c0315b0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 shutdown_connections 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55f0668210e0 0x55f066823500 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0667a9dc0 0x55f0667b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f0667a4c50 0x55f0667b37a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 --2- 172.21.15.31:0/3536006289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f0667a4260 0x55f0667b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe70b7fe640 0 client.16185 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 >> 172.21.15.31:0/3536006289 conn(0x55f06679b4c0 msgr2=0x55f06679e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 shutdown_connections 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 1 -- 172.21.15.31:0/3536006289 wait complete. 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7353c6740 20 client.16185 trim_cache size 0 max 16384 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:40:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe7140a2c30 msgr2=0x7fe7140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7326ba640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe7140a2c30 0x7fe7140a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe72409b4a0 tx=0x7fe724066370 comp rx=0 tx=0).stop 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 msgr2=0x7fe72c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.573+0000 7fe7326ba640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 0x7fe72c19eee0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe724009fd0 tx=0x7fe72400b260 comp rx=0 tx=0).stop 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 shutdown_connections 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe7140a2c30 0x7fe7140a50f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe72c102250 0x7fe72c19eee0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c101890 0x7fe72c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 --2- 172.21.15.31:0/2460513596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe72c1006a0 0x7fe72c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 >> 172.21.15.31:0/2460513596 conn(0x7fe72c0fbc10 msgr2=0x7fe72c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 shutdown_connections 2024-04-25T00:40:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:21.574+0000 7fe7326ba640 1 -- 172.21.15.31:0/2460513596 wait complete. 2024-04-25T00:40:22.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:22 smithi169 ceph-mon[33112]: pgmap v655: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:22 smithi040 ceph-mon[32945]: pgmap v655: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:22 smithi031 ceph-mon[29167]: pgmap v655: 97 pgs: 97 active+clean; 2.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:24.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:24 smithi169 ceph-mon[33112]: pgmap v656: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:40:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:24 smithi040 ceph-mon[32945]: pgmap v656: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:40:25.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:24 smithi031 ceph-mon[29167]: pgmap v656: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:40:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:26 smithi169 ceph-mon[33112]: pgmap v657: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:26 smithi040 ceph-mon[32945]: pgmap v657: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:27.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:26 smithi031 ceph-mon[29167]: pgmap v657: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:28 smithi169 ceph-mon[33112]: pgmap v658: 97 pgs: 97 active+clean; 2.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:28 smithi040 ceph-mon[32945]: pgmap v658: 97 pgs: 97 active+clean; 2.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:28 smithi031 ceph-mon[29167]: pgmap v658: 97 pgs: 97 active+clean; 2.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:30.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:30 smithi169 ceph-mon[33112]: pgmap v659: 97 pgs: 97 active+clean; 2.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:40:30.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:30 smithi040 ceph-mon[32945]: pgmap v659: 97 pgs: 97 active+clean; 2.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:40:31.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:30 smithi031 ceph-mon[29167]: pgmap v659: 97 pgs: 97 active+clean; 2.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:40:32.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:32 smithi169 ceph-mon[33112]: pgmap v660: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:32 smithi040 ceph-mon[32945]: pgmap v660: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:33.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:32 smithi031 ceph-mon[29167]: pgmap v660: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 -- 172.21.15.31:0/1148496540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c102280 msgr2=0x7fed7c108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 --2- 172.21.15.31:0/1148496540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c102280 0x7fed7c108b10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fed74009ca0 tx=0x7fed7402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 -- 172.21.15.31:0/1148496540 shutdown_connections 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 --2- 172.21.15.31:0/1148496540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c102280 0x7fed7c108b10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 --2- 172.21.15.31:0/1148496540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7c1018c0 0x7fed7c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 --2- 172.21.15.31:0/1148496540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c1006d0 0x7fed7c100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 -- 172.21.15.31:0/1148496540 >> 172.21.15.31:0/1148496540 conn(0x7fed7c0fbc40 msgr2=0x7fed7c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.674+0000 7fed82709640 1 -- 172.21.15.31:0/1148496540 shutdown_connections 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.675+0000 7fed82709640 1 -- 172.21.15.31:0/1148496540 wait complete. 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.675+0000 7fed82709640 1 Processor -- start 2024-04-25T00:40:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.675+0000 7fed82709640 1 -- start start 2024-04-25T00:40:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.675+0000 7fed82709640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 0x7fed7c10aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7c1018c0 0x7fed7c109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c102280 0x7fed7c109a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c10d320 con 0x7fed7c1006d0 2024-04-25T00:40:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c10d1a0 con 0x7fed7c1018c0 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c10d4a0 con 0x7fed7c102280 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 0x7fed7c10aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 0x7fed7c10aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46224/0 (socket says 172.21.15.31:46224) 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 -- 172.21.15.31:0/2961792438 learned_addr learned my addr 172.21.15.31:0/2961792438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed80ca6640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c102280 0x7fed7c109a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7c1018c0 0x7fed7c109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 -- 172.21.15.31:0/2961792438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c102280 msgr2=0x7fed7c109a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c102280 0x7fed7c109a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 -- 172.21.15.31:0/2961792438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7c1018c0 msgr2=0x7fed7c109510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7c1018c0 0x7fed7c109510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 -- 172.21.15.31:0/2961792438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c10a2e0 con 0x7fed7c1006d0 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed80ca6640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c102280 0x7fed7c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed7bfff640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 0x7fed7c10aed0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00d950 tx=0x7fed6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed797fa640 1 -- 172.21.15.31:0/2961792438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c017020 con 0x7fed7c1006d0 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed797fa640 1 -- 172.21.15.31:0/2961792438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed6c009c30 con 0x7fed7c1006d0 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c1a19f0 con 0x7fed7c1006d0 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed797fa640 1 -- 172.21.15.31:0/2961792438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00f6b0 con 0x7fed7c1006d0 2024-04-25T00:40:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.676+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c1a1f40 con 0x7fed7c1006d0 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed797fa640 1 -- 172.21.15.31:0/2961792438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fed6c01a020 con 0x7fed7c1006d0 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 Processor -- start 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 -- start start 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb205270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb20ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5641fb168cb0 con 0x5641fb205840 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5641fb1115b0 con 0x5641fb204e50 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed85410740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5641fb164c70 con 0x5641fb20a9b0 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed797fa640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed640a2c30 0x7fed640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46232/0 (socket says 172.21.15.31:46232) 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.679+0000 7fed7bfff640 1 -- 172.21.15.31:0/3436115986 learned_addr learned my addr 172.21.15.31:0/3436115986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed80ca6640 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb205270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb20ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed80ca6640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed640a2c30 0x7fed640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed797fa640 1 -- 172.21.15.31:0/2961792438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fed6c0ced90 con 0x7fed7c1006d0 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7bfff640 1 -- 172.21.15.31:0/3436115986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 msgr2=0x5641fb20ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb20ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7bfff640 1 -- 172.21.15.31:0/3436115986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 msgr2=0x5641fb205270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb205270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7bfff640 1 -- 172.21.15.31:0/3436115986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641fae7d210 con 0x5641fb205840 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed80ca6640 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb205270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20a340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009610 tx=0x7fed6c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e122c633e05965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed80ca6640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed640a2c30 0x7fed640a50f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed74031aa0 tx=0x7fed740095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed627fc640 1 -- 172.21.15.31:0/3436115986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c09bcc0 con 0x5641fb205840 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed627fc640 1 -- 172.21.15.31:0/3436115986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed6c009de0 con 0x5641fb205840 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.680+0000 7fed627fc640 1 -- 172.21.15.31:0/3436115986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c097c30 con 0x5641fb205840 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 -- 172.21.15.31:0/3436115986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 msgr2=0x5641fb20a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20a340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009610 tx=0x7fed6c022000 comp rx=0 tx=0).stop 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 -- 172.21.15.31:0/3436115986 shutdown_connections 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb20ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20a340 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 --2- 172.21.15.31:0/3436115986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb205270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 -- 172.21.15.31:0/3436115986 >> 172.21.15.31:0/3436115986 conn(0x5641fb2004c0 msgr2=0x5641fb2008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 -- 172.21.15.31:0/3436115986 shutdown_connections 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 -- 172.21.15.31:0/3436115986 wait complete. 2024-04-25T00:40:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.681+0000 7fed85410740 1 -- start start 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 Processor -- start 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb20f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb216780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5641fb20d900 con 0x5641fb205840 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5641fb124b50 con 0x5641fb204e50 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5641fb20da80 con 0x5641fb20a9b0 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed85410740 10 client.0 _ll_register_callbacks cb 0x5641fb15fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46246/0 (socket says 172.21.15.31:46246) 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 -- 172.21.15.31:0/3460345903 learned_addr learned my addr 172.21.15.31:0/3460345903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed80ca6640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb216780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb20f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 -- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 msgr2=0x5641fb216780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb216780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 -- 172.21.15.31:0/3460345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 msgr2=0x5641fb20f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb20f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 -- 172.21.15.31:0/3460345903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641faf49620 con 0x5641fb205840 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb20f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.682+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20f740 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fed70012700 tx=0x7fed70012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4514882979c72f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed5b7fe640 1 -- 172.21.15.31:0/3460345903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7000f8c0 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed5b7fe640 1 -- 172.21.15.31:0/3460345903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fed7000fa60 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6c009de0 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed5b7fe640 1 -- 172.21.15.31:0/3460345903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed7000fc60 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 10 client.16197 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5641fb219bc0 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed5b7fe640 1 -- 172.21.15.31:0/3460345903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed6c009de0 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed5b7fe640 1 -- 172.21.15.31:0/3460345903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed70025070 con 0x5641fb205840 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 10 client.16197 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 20 client.16197 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb21b290 0x5641fb21d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 4 client.16197 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.683+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641fb1761f0 con 0x5641fb21b290 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.684+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb21b290 0x5641fb21d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.684+0000 7fed7bfff640 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb21b290 0x5641fb21d700 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d104511920c2a31d server_cookie=bc7acbc3d1c7012e in_seq=0 out_seq=0 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.684+0000 7fed5b7fe640 10 client.16197 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed5b7fe640 1 -- 172.21.15.31:0/3460345903 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641fb1761f0 con 0x5641fb21b290 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed5b7fe640 10 client.16197 handle_command_reply: tid=0 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 client.16197 shutdown 2024-04-25T00:40:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 msgr2=0x5641fb20f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20f740 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fed70012700 tx=0x7fed70012bc0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 shutdown_connections 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb21b290 0x5641fb21d700 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb216780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb20f740 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 --2- 172.21.15.31:0/3460345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb20f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed5b7fe640 0 client.16197 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 >> 172.21.15.31:0/3460345903 conn(0x5641fb2004c0 msgr2=0x5641fb1ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 shutdown_connections 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 1 -- 172.21.15.31:0/3460345903 wait complete. 2024-04-25T00:40:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.685+0000 7fed85410740 20 client.16197 trim_cache size 0 max 16384 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.700+0000 7fed85410740 1 Processor -- start 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 -- start start 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb1fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb216740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb1ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1aae90 con 0x5641fb205840 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1ac9d0 con 0x5641fb204e50 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed85410740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1ab560 con 0x5641fb20a9b0 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb216740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb216740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46252/0 (socket says 172.21.15.31:46252) 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 -- 172.21.15.31:0/4123505562 learned_addr learned my addr 172.21.15.31:0/4123505562 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7bfff640 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb1fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 -- 172.21.15.31:0/4123505562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 msgr2=0x5641fb1ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb1ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 -- 172.21.15.31:0/4123505562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 msgr2=0x5641fb1fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb1fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 -- 172.21.15.31:0/4123505562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641fb201230 con 0x5641fb205840 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7bfff640 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb1fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb216740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fed70005f60 tx=0x7fed7000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630fd2eabb82a524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.701+0000 7fed61ffb640 1 -- 172.21.15.31:0/4123505562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7002b020 con 0x5641fb205840 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed61ffb640 1 -- 172.21.15.31:0/4123505562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed70022070 con 0x5641fb205840 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed61ffb640 1 -- 172.21.15.31:0/4123505562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7001c470 con 0x5641fb205840 2024-04-25T00:40:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 -- 172.21.15.31:0/4123505562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 msgr2=0x5641fb216740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb216740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fed70005f60 tx=0x7fed7000ff90 comp rx=0 tx=0).stop 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 -- 172.21.15.31:0/4123505562 shutdown_connections 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb20a9b0 0x5641fb1ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb205840 0x5641fb216740 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 --2- 172.21.15.31:0/4123505562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb204e50 0x5641fb1fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 -- 172.21.15.31:0/4123505562 >> 172.21.15.31:0/4123505562 conn(0x5641fb2004c0 msgr2=0x5641fb2008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 -- 172.21.15.31:0/4123505562 shutdown_connections 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 -- 172.21.15.31:0/4123505562 wait complete. 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.702+0000 7fed85410740 1 -- start start 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 Processor -- start 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 0x5641fb213f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb205840 0x5641fb2144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb20a9b0 0x5641fb215400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1ac300 con 0x5641fb204e50 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1abc30 con 0x5641fb20a9b0 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1a8e70 con 0x5641fb205840 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed85410740 10 client.0 _ll_register_callbacks cb 0x7fed70009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 0x5641fb213f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 0x5641fb213f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46256/0 (socket says 172.21.15.31:46256) 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 -- 172.21.15.31:0/4275819490 learned_addr learned my addr 172.21.15.31:0/4275819490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed80ca6640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb20a9b0 0x5641fb215400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb205840 0x5641fb2144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 -- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb205840 msgr2=0x5641fb2144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb205840 0x5641fb2144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 -- 172.21.15.31:0/4275819490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb20a9b0 msgr2=0x5641fb215400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb20a9b0 0x5641fb215400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7bfff640 1 -- 172.21.15.31:0/4275819490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641fb2159f0 con 0x5641fb204e50 2024-04-25T00:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.703+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb205840 0x5641fb2144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed7bfff640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 0x5641fb213f70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed6c097a20 tx=0x7fed6c09b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4eb0e9dd4707e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed5b7fe640 1 -- 172.21.15.31:0/4275819490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0b4060 con 0x5641fb204e50 2024-04-25T00:40:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed5b7fe640 1 -- 172.21.15.31:0/4275819490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fed6c097c30 con 0x5641fb204e50 2024-04-25T00:40:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed70022070 con 0x5641fb204e50 2024-04-25T00:40:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed5b7fe640 1 -- 172.21.15.31:0/4275819490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed6c014070 con 0x5641fb204e50 2024-04-25T00:40:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 10 client.16203 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5641fb2818f0 con 0x5641fb204e50 2024-04-25T00:40:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed5b7fe640 1 -- 172.21.15.31:0/4275819490 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed70022070 con 0x5641fb204e50 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed5b7fe640 1 -- 172.21.15.31:0/4275819490 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed6c024060 con 0x5641fb204e50 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 20 client.16203 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb281e00 0x5641fb2841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 4 client.16203 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.704+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641fb170540 con 0x5641fb281e00 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.705+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb281e00 0x5641fb2841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.705+0000 7fed5b7fe640 10 client.16203 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.705+0000 7fed7b7fe640 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb281e00 0x5641fb2841f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c660b7bcd5fd291 server_cookie=c3b9e505b3f15d58 in_seq=0 out_seq=0 2024-04-25T00:40:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed5b7fe640 1 -- 172.21.15.31:0/4275819490 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641fb170540 con 0x5641fb281e00 2024-04-25T00:40:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed5b7fe640 10 client.16203 handle_command_reply: tid=0 2024-04-25T00:40:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 client.16203 shutdown 2024-04-25T00:40:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 msgr2=0x5641fb213f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 0x5641fb213f70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed6c097a20 tx=0x7fed6c09b020 comp rx=0 tx=0).stop 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 shutdown_connections 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5641fb281e00 0x5641fb2841f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5641fb20a9b0 0x5641fb215400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed5b7fe640 0 client.16203 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5641fb205840 0x5641fb2144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 --2- 172.21.15.31:0/4275819490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641fb204e50 0x5641fb213f70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 >> 172.21.15.31:0/4275819490 conn(0x5641fb2004c0 msgr2=0x5641fb203440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 shutdown_connections 2024-04-25T00:40:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 1 -- 172.21.15.31:0/4275819490 wait complete. 2024-04-25T00:40:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed85410740 20 client.16203 trim_cache size 0 max 16384 2024-04-25T00:40:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:40:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:40:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:40:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed640a2c30 msgr2=0x7fed640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed640a2c30 0x7fed640a50f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed74031aa0 tx=0x7fed740095c0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 msgr2=0x7fed7c10aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 0x7fed7c10aed0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00d950 tx=0x7fed6c00de10 comp rx=0 tx=0).stop 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 shutdown_connections 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed7c102280 0x7fed7c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7c1018c0 0x7fed7c109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fed640a2c30 0x7fed640a50f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 --2- 172.21.15.31:0/2961792438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed7c1006d0 0x7fed7c10aed0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 >> 172.21.15.31:0/2961792438 conn(0x7fed7c0fbc40 msgr2=0x7fed7c104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 shutdown_connections 2024-04-25T00:40:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:34.723+0000 7fed82709640 1 -- 172.21.15.31:0/2961792438 wait complete. 2024-04-25T00:40:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:34 smithi169 ceph-mon[33112]: pgmap v661: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:34 smithi040 ceph-mon[32945]: pgmap v661: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:35.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:34 smithi031 ceph-mon[29167]: pgmap v661: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:40:36.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:36 smithi169 ceph-mon[33112]: pgmap v662: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:40:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:36 smithi040 ceph-mon[32945]: pgmap v662: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:40:37.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:36 smithi031 ceph-mon[29167]: pgmap v662: 97 pgs: 97 active+clean; 2.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:40:38.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:38 smithi169 ceph-mon[33112]: pgmap v663: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:40:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:38 smithi040 ceph-mon[32945]: pgmap v663: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:40:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:38 smithi031 ceph-mon[29167]: pgmap v663: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:40:40.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:40 smithi169 ceph-mon[33112]: pgmap v664: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:40.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:40 smithi040 ceph-mon[32945]: pgmap v664: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:41.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:40 smithi031 ceph-mon[29167]: pgmap v664: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:40:42.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:42 smithi169 ceph-mon[33112]: pgmap v665: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:42.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:42 smithi040 ceph-mon[32945]: pgmap v665: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:43.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:42 smithi031 ceph-mon[29167]: pgmap v665: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:40:44.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:44 smithi169 ceph-mon[33112]: pgmap v666: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:40:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:44 smithi040 ceph-mon[32945]: pgmap v666: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:40:45.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:44 smithi031 ceph-mon[29167]: pgmap v666: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:40:46.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:46 smithi169 ceph-mon[33112]: pgmap v667: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:46.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:46 smithi040 ceph-mon[32945]: pgmap v667: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:46 smithi031 ceph-mon[29167]: pgmap v667: 97 pgs: 97 active+clean; 2.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 -- 172.21.15.31:0/1074706729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 msgr2=0x7fc18c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 --2- 172.21.15.31:0/1074706729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c101dc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002a410 comp rx=0 tx=0).stop 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 -- 172.21.15.31:0/1074706729 shutdown_connections 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 --2- 172.21.15.31:0/1074706729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18c102300 0x7fc18c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 --2- 172.21.15.31:0/1074706729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c101dc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 --2- 172.21.15.31:0/1074706729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c100750 0x7fc18c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 -- 172.21.15.31:0/1074706729 >> 172.21.15.31:0/1074706729 conn(0x7fc18c0fbcc0 msgr2=0x7fc18c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 -- 172.21.15.31:0/1074706729 shutdown_connections 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 -- 172.21.15.31:0/1074706729 wait complete. 2024-04-25T00:40:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.823+0000 7fc1916ae640 1 Processor -- start 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 -- start start 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18c100750 0x7fc18c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c102300 0x7fc18c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c10c210 con 0x7fc18c101940 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c10c090 con 0x7fc18c102300 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc1916ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c10c390 con 0x7fc18c100750 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55360/0 (socket says 172.21.15.31:55360) 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3597383098 learned_addr learned my addr 172.21.15.31:0/3597383098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18affd640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18c100750 0x7fc18c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c102300 0x7fc18c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3597383098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18c100750 msgr2=0x7fc18c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18c100750 0x7fc18c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3597383098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c102300 msgr2=0x7fc18c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c102300 0x7fc18c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.824+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3597383098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c19f480 con 0x7fc18c101940 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c102300 0x7fc18c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c197da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc180004770 tx=0x7fc1800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc173fff640 1 -- 172.21.15.31:0/3597383098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18003f030 con 0x7fc18c101940 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc173fff640 1 -- 172.21.15.31:0/3597383098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc180003cb0 con 0x7fc18c101940 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c19f710 con 0x7fc18c101940 2024-04-25T00:40:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc173fff640 1 -- 172.21.15.31:0/3597383098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180035070 con 0x7fc18c101940 2024-04-25T00:40:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.825+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c19fc00 con 0x7fc18c101940 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.827+0000 7fc173fff640 1 -- 172.21.15.31:0/3597383098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc180042030 con 0x7fc18c101940 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 Processor -- start 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 -- start start 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f4e50 0x563b2f8f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f8fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563b2f858cb0 con 0x563b2f8f5840 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563b2f8015b0 con 0x563b2f8fa9b0 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc1943b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563b2f854c70 con 0x563b2f8f4e50 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc173fff640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1740a2c30 0x7fc1740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc18affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f8fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc18affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f8fa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55374/0 (socket says 172.21.15.31:55374) 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc18affd640 1 -- 172.21.15.31:0/3895347930 learned_addr learned my addr 172.21.15.31:0/3895347930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1740a2c30 0x7fc1740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc173fff640 1 -- 172.21.15.31:0/3597383098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc1800efab0 con 0x7fc18c101940 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.828+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f4e50 0x563b2f8f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18affd640 1 -- 172.21.15.31:0/3895347930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f4e50 msgr2=0x563b2f8f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18affd640 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f4e50 0x563b2f8f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18affd640 1 -- 172.21.15.31:0/3895347930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 msgr2=0x563b2f8fce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18affd640 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18affd640 1 -- 172.21.15.31:0/3895347930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2f56d210 con 0x563b2f8f5840 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18affd640 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f8fa340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00ac70 tx=0x7fc17c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e85462b85972ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1740a2c30 0x7fc1740a50f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x563b2f8f60a0 tx=0x7fc184009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc1717fa640 1 -- 172.21.15.31:0/3895347930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005220 con 0x563b2f8f5840 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc1717fa640 1 -- 172.21.15.31:0/3895347930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc17c0047a0 con 0x563b2f8f5840 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.829+0000 7fc1717fa640 1 -- 172.21.15.31:0/3895347930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c018780 con 0x563b2f8f5840 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- 172.21.15.31:0/3895347930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 msgr2=0x563b2f8fa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f8fa340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00ac70 tx=0x7fc17c00f790 comp rx=0 tx=0).stop 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- 172.21.15.31:0/3895347930 shutdown_connections 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f8fa340 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3895347930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f4e50 0x563b2f8f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- 172.21.15.31:0/3895347930 >> 172.21.15.31:0/3895347930 conn(0x563b2f8f04c0 msgr2=0x563b2f8f08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- 172.21.15.31:0/3895347930 shutdown_connections 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- 172.21.15.31:0/3895347930 wait complete. 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- start start 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 Processor -- start 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 0x563b2f8ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f5840 0x563b2f8ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f9067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563b2f8fd960 con 0x563b2f8f4e50 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563b2f814b50 con 0x563b2f8fa9b0 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.830+0000 7fc1943b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563b2f8fdae0 con 0x563b2f8f5840 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc1943b6740 10 client.0 _ll_register_callbacks cb 0x563b2f84fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 0x563b2f8ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 0x563b2f8ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55378/0 (socket says 172.21.15.31:55378) 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 -- 172.21.15.31:0/3001639256 learned_addr learned my addr 172.21.15.31:0/3001639256 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f9067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f5840 0x563b2f8ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 -- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f5840 msgr2=0x563b2f8ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f5840 0x563b2f8ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 -- 172.21.15.31:0/3001639256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 msgr2=0x563b2f9067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f9067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 -- 172.21.15.31:0/3001639256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2f639620 con 0x563b2f8f4e50 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f5840 0x563b2f8ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc18affd640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 0x563b2f8ff270 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00ac40 tx=0x7fc17c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869b12a4435fa505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc16eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc16eff5640 1 -- 172.21.15.31:0/3001639256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c027020 con 0x563b2f8f4e50 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc16eff5640 1 -- 172.21.15.31:0/3001639256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc17c005160 con 0x563b2f8f4e50 2024-04-25T00:40:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.831+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c0047a0 con 0x563b2f8f4e50 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc16eff5640 1 -- 172.21.15.31:0/3001639256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc17c032050 con 0x563b2f8f4e50 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 10 client.16215 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b2f909b60 con 0x563b2f8f4e50 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc16eff5640 1 -- 172.21.15.31:0/3001639256 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc17c0047a0 con 0x563b2f8f4e50 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc16eff5640 1 -- 172.21.15.31:0/3001639256 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc17c003b20 con 0x563b2f8f4e50 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 10 client.16215 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 20 client.16215 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f90afc0 0x563b2f90d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 4 client.16215 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b2f8661f0 con 0x563b2f90afc0 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.832+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f90afc0 0x563b2f90d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.833+0000 7fc16eff5640 10 client.16215 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.833+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f90afc0 0x563b2f90d3b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c575a06845310c1 server_cookie=4cdbfbc0496b5aef in_seq=0 out_seq=0 2024-04-25T00:40:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc16eff5640 1 -- 172.21.15.31:0/3001639256 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b2f8661f0 con 0x563b2f90afc0 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc16eff5640 10 client.16215 handle_command_reply: tid=0 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 client.16215 shutdown 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 msgr2=0x563b2f8ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 0x563b2f8ff270 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00ac40 tx=0x7fc17c0042f0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 shutdown_connections 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f90afc0 0x563b2f90d3b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8fa9b0 0x563b2f9067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8f5840 0x563b2f8ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3001639256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f4e50 0x563b2f8ff270 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 >> 172.21.15.31:0/3001639256 conn(0x563b2f8f04c0 msgr2=0x563b2f8ef5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc16eff5640 0 client.16215 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 shutdown_connections 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 1 -- 172.21.15.31:0/3001639256 wait complete. 2024-04-25T00:40:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.834+0000 7fc1943b6740 20 client.16215 trim_cache size 0 max 16384 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 Processor -- start 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 -- start start 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f8f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f9067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1a8b30 con 0x563b2f8f5840 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1aa670 con 0x563b2f8f4e50 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc1943b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1a9200 con 0x563b2f8fa9b0 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc18a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f9067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc18a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f9067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55382/0 (socket says 172.21.15.31:55382) 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.849+0000 7fc18a7fc640 1 -- 172.21.15.31:0/1737443101 learned_addr learned my addr 172.21.15.31:0/1737443101 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18affd640 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f8f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18a7fc640 1 -- 172.21.15.31:0/1737443101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 msgr2=0x563b2f8fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18a7fc640 1 -- 172.21.15.31:0/1737443101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 msgr2=0x563b2f8f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f8f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18a7fc640 1 -- 172.21.15.31:0/1737443101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2f8f9eb0 con 0x563b2f8f5840 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f9067b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc180003fd0 tx=0x7fc1800ed3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d8e9fcda71dc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc170ff9640 1 -- 172.21.15.31:0/1737443101 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180049040 con 0x563b2f8f5840 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc170ff9640 1 -- 172.21.15.31:0/1737443101 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1800c4ef0 con 0x563b2f8f5840 2024-04-25T00:40:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.850+0000 7fc170ff9640 1 -- 172.21.15.31:0/1737443101 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18003b460 con 0x563b2f8f5840 2024-04-25T00:40:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- 172.21.15.31:0/1737443101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 msgr2=0x563b2f9067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f9067b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc180003fd0 tx=0x7fc1800ed3f0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- 172.21.15.31:0/1737443101 shutdown_connections 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f8fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f9067b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- 172.21.15.31:0/1737443101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f8f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- 172.21.15.31:0/1737443101 >> 172.21.15.31:0/1737443101 conn(0x563b2f8f04c0 msgr2=0x563b2f8f3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- 172.21.15.31:0/1737443101 shutdown_connections 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- 172.21.15.31:0/1737443101 wait complete. 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- start start 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 Processor -- start 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f9035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f903b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f96f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1a9fa0 con 0x563b2f8f5840 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1a98d0 con 0x563b2f8f4e50 2024-04-25T00:40:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.851+0000 7fc1943b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1a6b10 con 0x563b2f8fa9b0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc1943b6740 10 client.0 _ll_register_callbacks cb 0x7fc17c00aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f903b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f903b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55394/0 (socket says 172.21.15.31:55394) 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3993947412 learned_addr learned my addr 172.21.15.31:0/3993947412 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18affd640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f9035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f96f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 msgr2=0x563b2f96f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f96f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3993947412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 msgr2=0x563b2f9035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f9035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 -- 172.21.15.31:0/3993947412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2f9041d0 con 0x563b2f8f5840 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18b7fe640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f96f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc18a7fc640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f903b30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc1800c49c0 tx=0x7fc180032a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bd331777d72cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc16eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:40:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc16eff5640 1 -- 172.21.15.31:0/3993947412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800d4020 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc16eff5640 1 -- 172.21.15.31:0/3993947412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc1800d5220 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1800c4ef0 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc16eff5640 1 -- 172.21.15.31:0/3993947412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1800ed630 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc1943b6740 10 client.16221 fetch_fsmap learned FSMap version 26 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.852+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b2f970bc0 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc16eff5640 1 -- 172.21.15.31:0/3993947412 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1800c4ef0 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc16eff5640 1 -- 172.21.15.31:0/3993947412 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc1800bc570 con 0x563b2f8f5840 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc1943b6740 10 client.16221 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc1943b6740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc1943b6740 20 client.16221 populate_metadata read hostname 'smithi031' 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f971540 0x563b2f973960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc1943b6740 4 client.16221 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b2f860540 con 0x563b2f971540 2024-04-25T00:40:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.853+0000 7fc18affd640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f971540 0x563b2f973960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.854+0000 7fc18affd640 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f971540 0x563b2f973960 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d1c639a7131127 server_cookie=6560068c711d7a58 in_seq=0 out_seq=0 2024-04-25T00:40:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.854+0000 7fc16eff5640 10 client.16221 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc16eff5640 1 -- 172.21.15.31:0/3993947412 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b2f860540 con 0x563b2f971540 2024-04-25T00:40:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc16eff5640 10 client.16221 handle_command_reply: tid=0 2024-04-25T00:40:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 client.16221 shutdown 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 msgr2=0x563b2f903b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f903b30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc1800c49c0 tx=0x7fc180032a60 comp rx=0 tx=0).stop 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 shutdown_connections 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x563b2f971540 0x563b2f973960 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563b2f8fa9b0 0x563b2f96f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b2f8f5840 0x563b2f903b30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 --2- 172.21.15.31:0/3993947412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b2f8f4e50 0x563b2f9035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc16eff5640 0 client.16221 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 >> 172.21.15.31:0/3993947412 conn(0x563b2f8f04c0 msgr2=0x563b2f8f3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 shutdown_connections 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.874+0000 7fc1943b6740 1 -- 172.21.15.31:0/3993947412 wait complete. 2024-04-25T00:40:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1943b6740 20 client.16221 trim_cache size 0 max 16384 2024-04-25T00:40:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:40:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:40:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:40:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1740a2c30 msgr2=0x7fc1740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1740a2c30 0x7fc1740a50f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x563b2f8f60a0 tx=0x7fc184009530 comp rx=0 tx=0).stop 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 msgr2=0x7fc18c197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c197da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc180004770 tx=0x7fc1800042f0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 shutdown_connections 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18c102300 0x7fc18c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fc1740a2c30 0x7fc1740a50f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc18c101940 0x7fc18c197da0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 --2- 172.21.15.31:0/3597383098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18c100750 0x7fc18c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 >> 172.21.15.31:0/3597383098 conn(0x7fc18c0fbcc0 msgr2=0x7fc18c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 shutdown_connections 2024-04-25T00:40:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:40:47.875+0000 7fc1916ae640 1 -- 172.21.15.31:0/3597383098 wait complete. 2024-04-25T00:40:48.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:48 smithi040 ceph-mon[32945]: pgmap v668: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:40:49.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:48 smithi031 ceph-mon[29167]: pgmap v668: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:40:49.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:48 smithi169 ceph-mon[33112]: pgmap v668: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:40:50.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:50 smithi040 ceph-mon[32945]: pgmap v669: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-25T00:40:51.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:50 smithi031 ceph-mon[29167]: pgmap v669: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-25T00:40:51.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:50 smithi169 ceph-mon[33112]: pgmap v669: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-25T00:40:53.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:52 smithi031 ceph-mon[29167]: pgmap v670: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T00:40:53.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:52 smithi169 ceph-mon[33112]: pgmap v670: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T00:40:53.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:52 smithi040 ceph-mon[32945]: pgmap v670: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T00:40:54.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:54 smithi169 ceph-mon[33112]: pgmap v671: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:40:54.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:54 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:40:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:54 smithi040 ceph-mon[32945]: pgmap v671: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:40:54.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:54 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:40:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:54 smithi031 ceph-mon[29167]: pgmap v671: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:40:54.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:54 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: pgmap v672: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:40:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:40:57.273 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: pgmap v672: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:40:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:40:57.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: pgmap v672: 97 pgs: 97 active+clean; 2.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:40:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:57.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:40:57.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:40:57.396 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:40:59.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:40:58 smithi169 ceph-mon[33112]: pgmap v673: 97 pgs: 97 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:59.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:40:58 smithi031 ceph-mon[29167]: pgmap v673: 97 pgs: 97 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:40:59.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:40:58 smithi040 ceph-mon[32945]: pgmap v673: 97 pgs: 97 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/2534954943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a74102280 msgr2=0x7f4a74108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/2534954943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a74102280 0x7f4a74108b10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/2534954943 shutdown_connections 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/2534954943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a74102280 0x7f4a74108b10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/2534954943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a741018c0 0x7f4a74101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/2534954943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a741006d0 0x7f4a74100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/2534954943 >> 172.21.15.31:0/2534954943 conn(0x7f4a740fbc40 msgr2=0x7f4a740fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/2534954943 shutdown_connections 2024-04-25T00:41:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.975+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/2534954943 wait complete. 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 Processor -- start 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 -- start start 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 0x7f4a741978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a741018c0 0x7f4a74197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74102280 0x7f4a7419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410c200 con 0x7f4a741006d0 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410c080 con 0x7f4a741018c0 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a7b5b7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410c380 con 0x7f4a74102280 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 0x7f4a741978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 0x7f4a741978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39976/0 (socket says 172.21.15.31:39976) 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 -- 172.21.15.31:0/356847015 learned_addr learned my addr 172.21.15.31:0/356847015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79b54640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74102280 0x7f4a7419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a78b52640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a741018c0 0x7f4a74197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 -- 172.21.15.31:0/356847015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74102280 msgr2=0x7f4a7419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74102280 0x7f4a7419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 -- 172.21.15.31:0/356847015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a741018c0 msgr2=0x7f4a74197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a741018c0 0x7f4a74197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.976+0000 7f4a79353640 1 -- 172.21.15.31:0/356847015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7419f4f0 con 0x7f4a741006d0 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.977+0000 7f4a79353640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 0x7f4a741978d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000d770 tx=0x7f4a7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.977+0000 7f4a6a7fc640 1 -- 172.21.15.31:0/356847015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70017070 con 0x7f4a741006d0 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.977+0000 7f4a6a7fc640 1 -- 172.21.15.31:0/356847015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a70009c30 con 0x7f4a741006d0 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.977+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7419f7e0 con 0x7f4a741006d0 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.977+0000 7f4a6a7fc640 1 -- 172.21.15.31:0/356847015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70003d00 con 0x7f4a741006d0 2024-04-25T00:41:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.977+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7419fd80 con 0x7f4a741006d0 2024-04-25T00:41:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.979+0000 7f4a6a7fc640 1 -- 172.21.15.31:0/356847015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4a70003ea0 con 0x7f4a741006d0 2024-04-25T00:41:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 Processor -- start 2024-04-25T00:41:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 -- start start 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 0x5649d0e9e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0ea3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0ea5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5649d0e01cb0 con 0x5649d0ea39b0 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5649d0daa5b0 con 0x5649d0e9e840 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649d0dfdc70 con 0x5649d0e9de50 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a6a7fc640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a600a2cb0 0x7f4a600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a78b52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0ea5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a79b54640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 0x5649d0e9e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 0x5649d0e9e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:41464/0 (socket says 172.21.15.31:41464) 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.980+0000 7f4a79353640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0ea3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 -- 172.21.15.31:0/1723084127 learned_addr learned my addr 172.21.15.31:0/1723084127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a600a2cb0 0x7f4a600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a6a7fc640 1 -- 172.21.15.31:0/356847015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a700ced90 con 0x7f4a741006d0 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 -- 172.21.15.31:0/1723084127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 msgr2=0x5649d0ea3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0ea3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 -- 172.21.15.31:0/1723084127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 msgr2=0x5649d0ea5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0ea5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 -- 172.21.15.31:0/1723084127 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d0b16210 con 0x5649d0e9de50 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a78b52640 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0ea5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.981+0000 7f4a79b54640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a600a2cb0 0x7f4a600a5170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c03b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.982+0000 7f4a79b54640 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 0x5649d0e9e270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c004af0 tx=0x7f4a6c032d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e59ffbf53fe39c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.982+0000 7f4a5f7fe640 1 -- 172.21.15.31:0/1723084127 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c04f070 con 0x5649d0e9de50 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.982+0000 7f4a5f7fe640 1 -- 172.21.15.31:0/1723084127 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c036040 con 0x5649d0e9de50 2024-04-25T00:41:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.982+0000 7f4a5f7fe640 1 -- 172.21.15.31:0/1723084127 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c030930 con 0x5649d0e9de50 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/1723084127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 msgr2=0x5649d0e9e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 0x5649d0e9e270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c004af0 tx=0x7f4a6c032d90 comp rx=0 tx=0).stop 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/1723084127 shutdown_connections 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0ea5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0ea3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/1723084127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9de50 0x5649d0e9e270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/1723084127 >> 172.21.15.31:0/1723084127 conn(0x5649d0e994c0 msgr2=0x5649d0e998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/1723084127 shutdown_connections 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/1723084127 wait complete. 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.983+0000 7f4a7e3a2740 1 -- start start 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 Processor -- start 2024-04-25T00:41:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5649d0ea6780 con 0x5649d0ea39b0 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5649d0dbdb50 con 0x5649d0e9de50 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649d0ea6900 con 0x5649d0e9e840 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a7e3a2740 10 client.0 _ll_register_callbacks cb 0x5649d0df8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a79b54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a79353640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a78b52640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a79b54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39988/0 (socket says 172.21.15.31:39988) 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a79b54640 1 -- 172.21.15.31:0/3520421502 learned_addr learned my addr 172.21.15.31:0/3520421502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.984+0000 7f4a79353640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:45454/0 (socket says 172.21.15.31:45454) 2024-04-25T00:41:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79b54640 1 -- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9e840 msgr2=0x5649d0ea8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79b54640 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79b54640 1 -- 172.21.15.31:0/3520421502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 msgr2=0x5649d0ea80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79b54640 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79b54640 1 -- 172.21.15.31:0/3520421502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d0be2620 con 0x5649d0ea39b0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79353640 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a79b54640 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c02a900 tx=0x7f4a6c004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ea67bf20da14d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a5cff9640 1 -- 172.21.15.31:0/3520421502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c04f040 con 0x5649d0ea39b0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a5cff9640 1 -- 172.21.15.31:0/3520421502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a6c030830 con 0x5649d0ea39b0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c036040 con 0x5649d0ea39b0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a5cff9640 1 -- 172.21.15.31:0/3520421502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c004040 con 0x5649d0ea39b0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a7e3a2740 10 client.16233 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.985+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649d0ea6780 con 0x5649d0ea39b0 2024-04-25T00:41:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a5cff9640 1 -- 172.21.15.31:0/3520421502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c036040 con 0x5649d0ea39b0 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a5cff9640 1 -- 172.21.15.31:0/3520421502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a6c04bab0 con 0x5649d0ea39b0 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a7e3a2740 10 client.16233 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a7e3a2740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a7e3a2740 20 client.16233 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0eb3e20 0x5649d0eb6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a7e3a2740 4 client.16233 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d0e0f1f0 con 0x5649d0eb3e20 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a79353640 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0eb3e20 0x5649d0eb6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a79353640 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0eb3e20 0x5649d0eb6210 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97d3d7c94ae3d20 server_cookie=cfa0204a74d2da7f in_seq=0 out_seq=0 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.986+0000 7f4a5cff9640 10 client.16233 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a5cff9640 1 -- 172.21.15.31:0/3520421502 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649d0e0f1f0 con 0x5649d0eb3e20 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a5cff9640 10 client.16233 handle_command_reply: tid=0 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 client.16233 shutdown 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 msgr2=0x5649d0eaf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c02a900 tx=0x7f4a6c004720 comp rx=0 tx=0).stop 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 shutdown_connections 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0eb3e20 0x5649d0eb6210 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf640 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a5cff9640 0 client.16233 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/3520421502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.987+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 >> 172.21.15.31:0/3520421502 conn(0x5649d0e994c0 msgr2=0x5649d0e985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.988+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 shutdown_connections 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.988+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/3520421502 wait complete. 2024-04-25T00:41:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:00.988+0000 7f4a7e3a2740 20 client.16233 trim_cache size 0 max 16384 2024-04-25T00:41:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 Processor -- start 2024-04-25T00:41:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 -- start start 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741a8c00 con 0x5649d0e9e840 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741aa740 con 0x5649d0e9de50 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741a92d0 con 0x5649d0ea39b0 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a78b52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a78b52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39992/0 (socket says 172.21.15.31:39992) 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a78b52640 1 -- 172.21.15.31:0/2178014102 learned_addr learned my addr 172.21.15.31:0/2178014102 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a79b54640 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.003+0000 7f4a79353640 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a78b52640 1 -- 172.21.15.31:0/2178014102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 msgr2=0x5649d0eaf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a78b52640 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a78b52640 1 -- 172.21.15.31:0/2178014102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 msgr2=0x5649d0ea80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a78b52640 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a78b52640 1 -- 172.21.15.31:0/2178014102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d0e9a230 con 0x5649d0e9e840 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a79353640 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a78b52640 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400bcb0 tx=0x7f4a64012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48779e3a288cbbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a5effd640 1 -- 172.21.15.31:0/2178014102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400bda0 con 0x5649d0e9e840 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a5effd640 1 -- 172.21.15.31:0/2178014102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a640118f0 con 0x5649d0e9e840 2024-04-25T00:41:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.004+0000 7f4a5effd640 1 -- 172.21.15.31:0/2178014102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400f960 con 0x5649d0e9e840 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/2178014102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 msgr2=0x5649d0ea8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400bcb0 tx=0x7f4a64012820 comp rx=0 tx=0).stop 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/2178014102 shutdown_connections 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0eaf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9e840 0x5649d0ea8600 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/2178014102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9de50 0x5649d0ea80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/2178014102 >> 172.21.15.31:0/2178014102 conn(0x5649d0e994c0 msgr2=0x5649d0e998a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/2178014102 shutdown_connections 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/2178014102 wait complete. 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- start start 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 Processor -- start 2024-04-25T00:41:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 0x5649d0eac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0eac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0f18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741aa070 con 0x5649d0e9de50 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741a99a0 con 0x5649d0e9e840 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741a6be0 con 0x5649d0ea39b0 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.005+0000 7f4a7e3a2740 10 client.0 _ll_register_callbacks cb 0x7f4a70095f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 0x5649d0eac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 0x5649d0eac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40008/0 (socket says 172.21.15.31:40008) 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 -- 172.21.15.31:0/377480360 learned_addr learned my addr 172.21.15.31:0/377480360 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a78b52640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0eac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79b54640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0f18380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 -- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 msgr2=0x5649d0f18380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0f18380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 -- 172.21.15.31:0/377480360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 msgr2=0x5649d0eac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0eac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 -- 172.21.15.31:0/377480360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d0eaf840 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a79353640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 0x5649d0eac280 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a7009b260 tx=0x7f4a70023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac64a7f4570d6833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a5cff9640 1 -- 172.21.15.31:0/377480360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70020040 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a5cff9640 1 -- 172.21.15.31:0/377480360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4a700cc910 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a640118f0 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a5cff9640 1 -- 172.21.15.31:0/377480360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a70009de0 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a7e3a2740 10 client.16239 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.006+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649d0f19800 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a5cff9640 1 -- 172.21.15.31:0/377480360 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a640118f0 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a5cff9640 1 -- 172.21.15.31:0/377480360 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a700a3d40 con 0x5649d0e9de50 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a7e3a2740 10 client.16239 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a7e3a2740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a7e3a2740 20 client.16239 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0f1a130 0x5649d0f1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a7e3a2740 4 client.16239 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649d0e09540 con 0x5649d0f1a130 2024-04-25T00:41:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a78b52640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0f1a130 0x5649d0f1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a78b52640 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0f1a130 0x5649d0f1c550 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf78c2eec4ac0b64 server_cookie=5a2754eaafd43001 in_seq=0 out_seq=0 2024-04-25T00:41:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.007+0000 7f4a5cff9640 10 client.16239 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a5cff9640 1 -- 172.21.15.31:0/377480360 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649d0e09540 con 0x5649d0f1a130 2024-04-25T00:41:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a5cff9640 10 client.16239 handle_command_reply: tid=0 2024-04-25T00:41:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 client.16239 shutdown 2024-04-25T00:41:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 msgr2=0x5649d0eac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 0x5649d0eac280 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a7009b260 tx=0x7f4a70023000 comp rx=0 tx=0).stop 2024-04-25T00:41:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 shutdown_connections 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x5649d0f1a130 0x5649d0f1c550 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649d0ea39b0 0x5649d0f18380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649d0e9e840 0x5649d0eac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a5cff9640 0 client.16239 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 --2- 172.21.15.31:0/377480360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649d0e9de50 0x5649d0eac280 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 >> 172.21.15.31:0/377480360 conn(0x5649d0e994c0 msgr2=0x5649d0ea80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 shutdown_connections 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 1 -- 172.21.15.31:0/377480360 wait complete. 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.024+0000 7f4a7e3a2740 20 client.16239 trim_cache size 0 max 16384 2024-04-25T00:41:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:41:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:41:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:41:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:41:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a600a2cb0 msgr2=0x7f4a600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a600a2cb0 0x7f4a600a5170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c03b000 comp rx=0 tx=0).stop 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 msgr2=0x7f4a741978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 0x7f4a741978d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000d770 tx=0x7f4a7000dc30 comp rx=0 tx=0).stop 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 shutdown_connections 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74102280 0x7f4a7419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a741018c0 0x7f4a74197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4a600a2cb0 0x7f4a600a5170 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 --2- 172.21.15.31:0/356847015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a741006d0 0x7f4a741978d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 >> 172.21.15.31:0/356847015 conn(0x7f4a740fbc40 msgr2=0x7f4a740fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 shutdown_connections 2024-04-25T00:41:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:01.025+0000 7f4a7b5b7640 1 -- 172.21.15.31:0/356847015 wait complete. 2024-04-25T00:41:01.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:00 smithi040 ceph-mon[32945]: pgmap v674: 97 pgs: 97 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:41:01.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:00 smithi031 ceph-mon[29167]: pgmap v674: 97 pgs: 97 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:41:01.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:00 smithi169 ceph-mon[33112]: pgmap v674: 97 pgs: 97 active+clean; 3.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-25T00:41:03.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:02 smithi031 ceph-mon[29167]: pgmap v675: 97 pgs: 97 active+clean; 3.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:41:03.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:02 smithi169 ceph-mon[33112]: pgmap v675: 97 pgs: 97 active+clean; 3.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:41:03.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:02 smithi040 ceph-mon[32945]: pgmap v675: 97 pgs: 97 active+clean; 3.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T00:41:05.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:04 smithi031 ceph-mon[29167]: pgmap v676: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:41:05.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:04 smithi169 ceph-mon[33112]: pgmap v676: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:41:05.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:04 smithi040 ceph-mon[32945]: pgmap v676: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-04-25T00:41:07.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:06 smithi031 ceph-mon[29167]: pgmap v677: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:41:07.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:06 smithi169 ceph-mon[33112]: pgmap v677: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:41:07.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:06 smithi040 ceph-mon[32945]: pgmap v677: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:41:09.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:08 smithi031 ceph-mon[29167]: pgmap v678: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:09.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:08 smithi169 ceph-mon[33112]: pgmap v678: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:09.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:08 smithi040 ceph-mon[32945]: pgmap v678: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:11.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:10 smithi031 ceph-mon[29167]: pgmap v679: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:11.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:10 smithi169 ceph-mon[33112]: pgmap v679: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:11.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:10 smithi040 ceph-mon[32945]: pgmap v679: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:499/600: open fd = 3 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:500/600: open fd = 3 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:501/600: open fd = 3 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.091 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:502/600: open fd = 3 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:503/600: open fd = 3 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:504/600: open fd = 3 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.092 INFO:tasks.workunit.client.0.smithi031.stdout:505/600: open fd = 3 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:506/600: open fd = 3 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:507/600: open fd = 3 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:508/600: open fd = 3 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:509/600: open fd = 3 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:510/600: open fd = 3 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:511/600: open fd = 3 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:512/600: open fd = 3 2024-04-25T00:41:13.094 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:513/600: open fd = 3 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:514/600: open fd = 3 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:515/600: open fd = 3 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.095 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:516/600: open fd = 3 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:517/600: open fd = 3 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:518/600: open fd = 3 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:519/600: open fd = 3 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.096 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:520/600: open fd = 3 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:521/600: open fd = 3 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:522/600: open fd = 3 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:523/600: open fd = 3 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:524/600: open fd = 3 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:525/600: open fd = 3 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.098 INFO:tasks.workunit.client.0.smithi031.stdout:526/600: open fd = 3 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:527/600: open fd = 3 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:528/600: open fd = 3 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:529/600: open fd = 3 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.099 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:530/600: open fd = 3 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:531/600: open fd = 3 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:532/600: open fd = 3 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:533/600: open fd = 3 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.100 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:534/600: open fd = 3 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:535/600: open fd = 3 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:536/600: open fd = 3 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:537/600: open fd = 3 2024-04-25T00:41:13.101 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:538/600: open fd = 3 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:539/600: open fd = 3 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:540/600: open fd = 3 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.102 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:541/600: open fd = 3 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:542/600: open fd = 3 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:543/600: open fd = 3 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:544/600: open fd = 3 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.103 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:545/600: open fd = 3 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:546/600: open fd = 3 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:547/600: open fd = 3 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:548/600: open fd = 3 2024-04-25T00:41:13.104 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:549/600: open fd = 3 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:550/600: open fd = 3 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:551/600: open fd = 3 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:552/600: open fd = 3 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:553/600: open fd = 3 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:554/600: open fd = 3 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:555/600: open fd = 3 2024-04-25T00:41:13.106 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:556/600: open fd = 3 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:557/600: open fd = 3 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:558/600: open fd = 3 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.107 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:559/600: open fd = 3 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:560/600: open fd = 3 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:13.108 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:13.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:12 smithi031 ceph-mon[29167]: pgmap v680: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:13.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:12 smithi169 ceph-mon[33112]: pgmap v680: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:13.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:12 smithi040 ceph-mon[32945]: pgmap v680: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 -- 172.21.15.31:0/335038082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea50103380 msgr2=0x7fea50109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 --2- 172.21.15.31:0/335038082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea50103380 0x7fea50109c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fea48009ca0 tx=0x7fea4802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 -- 172.21.15.31:0/335038082 shutdown_connections 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 --2- 172.21.15.31:0/335038082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea50103380 0x7fea50109c10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 --2- 172.21.15.31:0/335038082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea501029c0 0x7fea50102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 --2- 172.21.15.31:0/335038082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 0x7fea50101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 -- 172.21.15.31:0/335038082 >> 172.21.15.31:0/335038082 conn(0x7fea500fcd60 msgr2=0x7fea500ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 -- 172.21.15.31:0/335038082 shutdown_connections 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.125+0000 7fea57dc1640 1 -- 172.21.15.31:0/335038082 wait complete. 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 Processor -- start 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 -- start start 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 0x7fea500754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 0x7fea50073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea50103380 0x7fea50074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea5010d300 con 0x7fea501029c0 2024-04-25T00:41:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea5010d180 con 0x7fea501017d0 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea57dc1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea5010d480 con 0x7fea50103380 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea5535c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 0x7fea50073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea5535c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 0x7fea50073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60452/0 (socket says 172.21.15.31:60452) 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea5535c640 1 -- 172.21.15.31:0/1104840342 learned_addr learned my addr 172.21.15.31:0/1104840342 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.126+0000 7fea55b5d640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 0x7fea500754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5635e640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea50103380 0x7fea50074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5535c640 1 -- 172.21.15.31:0/1104840342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea50103380 msgr2=0x7fea50074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5535c640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea50103380 0x7fea50074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5535c640 1 -- 172.21.15.31:0/1104840342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 msgr2=0x7fea500754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5535c640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 0x7fea500754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5535c640 1 -- 172.21.15.31:0/1104840342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea50074900 con 0x7fea501029c0 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea55b5d640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 0x7fea500754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea5535c640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 0x7fea50073b30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d770 tx=0x7fea4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea46ffd640 1 -- 172.21.15.31:0/1104840342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40017070 con 0x7fea501029c0 2024-04-25T00:41:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea46ffd640 1 -- 172.21.15.31:0/1104840342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea400054d0 con 0x7fea501029c0 2024-04-25T00:41:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea501a1a00 con 0x7fea501029c0 2024-04-25T00:41:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea46ffd640 1 -- 172.21.15.31:0/1104840342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40004cf0 con 0x7fea501029c0 2024-04-25T00:41:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.127+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501a1f00 con 0x7fea501029c0 2024-04-25T00:41:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.129+0000 7fea46ffd640 1 -- 172.21.15.31:0/1104840342 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea40004e90 con 0x7fea501029c0 2024-04-25T00:41:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 Processor -- start 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 -- start start 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 0x55fc81c06270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c0de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fc81b69cb0 con 0x55fc81c05e50 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fc81b125b0 con 0x55fc81c0b9b0 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.130+0000 7fea5aba9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc81b65c70 con 0x55fc81c06840 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 0x55fc81c06270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 0x55fc81c06270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60454/0 (socket says 172.21.15.31:60454) 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 -- 172.21.15.31:0/3156398477 learned_addr learned my addr 172.21.15.31:0/3156398477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea46ffd640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fea3c0a2c30 0x7fea3c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5535c640 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fea3c0a2c30 0x7fea3c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 -- 172.21.15.31:0/3156398477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 msgr2=0x55fc81c0b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c0b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 -- 172.21.15.31:0/3156398477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 msgr2=0x55fc81c0de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c0de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 -- 172.21.15.31:0/3156398477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc8187e210 con 0x55fc81c05e50 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea46ffd640 1 -- 172.21.15.31:0/1104840342 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fea400ceb20 con 0x7fea501029c0 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 0x55fc81c06270 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fea48035000 tx=0x7fea48031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dd4ee4855b190d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea5635e640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fea3c0a2c30 0x7fea3c0a50f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea48031770 tx=0x7fea48049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.131+0000 7fea3bfff640 1 -- 172.21.15.31:0/3156398477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48004440 con 0x55fc81c05e50 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea3bfff640 1 -- 172.21.15.31:0/3156398477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48036040 con 0x55fc81c05e50 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea3bfff640 1 -- 172.21.15.31:0/3156398477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48030ed0 con 0x55fc81c05e50 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 -- 172.21.15.31:0/3156398477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 msgr2=0x55fc81c06270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 0x55fc81c06270 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fea48035000 tx=0x7fea48031700 comp rx=0 tx=0).stop 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 -- 172.21.15.31:0/3156398477 shutdown_connections 2024-04-25T00:41:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c0de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c0b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3156398477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c05e50 0x55fc81c06270 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 -- 172.21.15.31:0/3156398477 >> 172.21.15.31:0/3156398477 conn(0x55fc81c014c0 msgr2=0x55fc81c018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 -- 172.21.15.31:0/3156398477 shutdown_connections 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.132+0000 7fea5aba9740 1 -- 172.21.15.31:0/3156398477 wait complete. 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 -- start start 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 Processor -- start 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c05e50 0x55fc81c102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c17850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fc81c0e900 con 0x55fc81c0b9b0 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fc81b25b50 con 0x55fc81c05e50 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc81c0ea80 con 0x55fc81c06840 2024-04-25T00:41:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5aba9740 10 client.0 _ll_register_callbacks cb 0x55fc81b60ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.133+0000 7fea5635e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c17850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c17850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60462/0 (socket says 172.21.15.31:60462) 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 -- 172.21.15.31:0/3555943540 learned_addr learned my addr 172.21.15.31:0/3555943540 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5535c640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c05e50 0x55fc81c102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 -- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 msgr2=0x55fc81c10810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c10810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 -- 172.21.15.31:0/3555943540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c05e50 msgr2=0x55fc81c102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c05e50 0x55fc81c102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 -- 172.21.15.31:0/3555943540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc8194a620 con 0x55fc81c0b9b0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5635e640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c17850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fea48031aa0 tx=0x7fea48009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b23bdb1a280edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea397fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea397fa640 1 -- 172.21.15.31:0/3555943540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48060060 con 0x55fc81c0b9b0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea48036040 con 0x55fc81c0b9b0 2024-04-25T00:41:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea397fa640 1 -- 172.21.15.31:0/3555943540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fea48003e00 con 0x55fc81c0b9b0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea397fa640 1 -- 172.21.15.31:0/3555943540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea48030c10 con 0x55fc81c0b9b0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5aba9740 10 client.16251 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.134+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc81c0e900 con 0x55fc81c0b9b0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea397fa640 1 -- 172.21.15.31:0/3555943540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48036040 con 0x55fc81c0b9b0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea397fa640 1 -- 172.21.15.31:0/3555943540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea4802f550 con 0x55fc81c0b9b0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea5aba9740 10 client.16251 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea5aba9740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea5aba9740 20 client.16251 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c1e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea5aba9740 4 client.16251 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc81b771f0 con 0x55fc81c1bf30 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.135+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c1e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.136+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c1e320 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47ed378f2e3c87e server_cookie=b1f0168acaf268e4 in_seq=0 out_seq=0 2024-04-25T00:41:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.136+0000 7fea397fa640 10 client.16251 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea397fa640 1 -- 172.21.15.31:0/3555943540 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc81b771f0 con 0x55fc81c1bf30 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea397fa640 10 client.16251 handle_command_reply: tid=0 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 client.16251 shutdown 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 msgr2=0x55fc81c17850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c17850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fea48031aa0 tx=0x7fea48009c40 comp rx=0 tx=0).stop 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 shutdown_connections 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c1e320 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea397fa640 0 client.16251 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c17850 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c06840 0x55fc81c10810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3555943540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c05e50 0x55fc81c102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 >> 172.21.15.31:0/3555943540 conn(0x55fc81c014c0 msgr2=0x55fc81c005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 shutdown_connections 2024-04-25T00:41:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 1 -- 172.21.15.31:0/3555943540 wait complete. 2024-04-25T00:41:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.137+0000 7fea5aba9740 20 client.16251 trim_cache size 0 max 16384 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 Processor -- start 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 -- start start 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81bffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c06840 0x55fc81c17810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea501aad60 con 0x55fc81c0b9b0 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea501ac8a0 con 0x55fc81c06840 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5aba9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea501ab430 con 0x55fc81c05e50 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5635e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c00270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5635e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c00270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60464/0 (socket says 172.21.15.31:60464) 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.152+0000 7fea5635e640 1 -- 172.21.15.31:0/2013026445 learned_addr learned my addr 172.21.15.31:0/2013026445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea55b5d640 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81bffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5535c640 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c06840 0x55fc81c17810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5635e640 1 -- 172.21.15.31:0/2013026445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 msgr2=0x55fc81bffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5635e640 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81bffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5635e640 1 -- 172.21.15.31:0/2013026445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c06840 msgr2=0x55fc81c17810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5635e640 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c06840 0x55fc81c17810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5635e640 1 -- 172.21.15.31:0/2013026445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc81c0e5b0 con 0x55fc81c0b9b0 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5535c640 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c06840 0x55fc81c17810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea5635e640 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c00270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fea4802f990 tx=0x7fea48031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aed34e52abfd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea3b7fe640 1 -- 172.21.15.31:0/2013026445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4806a020 con 0x55fc81c0b9b0 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea3b7fe640 1 -- 172.21.15.31:0/2013026445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48036030 con 0x55fc81c0b9b0 2024-04-25T00:41:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.153+0000 7fea3b7fe640 1 -- 172.21.15.31:0/2013026445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48060040 con 0x55fc81c0b9b0 2024-04-25T00:41:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 -- 172.21.15.31:0/2013026445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 msgr2=0x55fc81c00270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c00270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fea4802f990 tx=0x7fea48031700 comp rx=0 tx=0).stop 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 -- 172.21.15.31:0/2013026445 shutdown_connections 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c0b9b0 0x55fc81c00270 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c06840 0x55fc81c17810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 --2- 172.21.15.31:0/2013026445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81bffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 -- 172.21.15.31:0/2013026445 >> 172.21.15.31:0/2013026445 conn(0x55fc81c014c0 msgr2=0x55fc81c07ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 -- 172.21.15.31:0/2013026445 shutdown_connections 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 -- 172.21.15.31:0/2013026445 wait complete. 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 -- start start 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.154+0000 7fea5aba9740 1 Processor -- start 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81c14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 0x55fc81c15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c16450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea501ac1d0 con 0x55fc81c06840 2024-04-25T00:41:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea501abb00 con 0x55fc81c0b9b0 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a8d40 con 0x55fc81c05e50 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 10 client.0 _ll_register_callbacks cb 0x7fea4009b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 0x55fc81c15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 0x55fc81c15520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60478/0 (socket says 172.21.15.31:60478) 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 -- 172.21.15.31:0/3733353399 learned_addr learned my addr 172.21.15.31:0/3733353399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81c14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5635e640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c16450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 -- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 msgr2=0x55fc81c14fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81c14fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 -- 172.21.15.31:0/3733353399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 msgr2=0x55fc81c16450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c16450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 -- 172.21.15.31:0/3733353399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc81c16a40 con 0x55fc81c06840 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5635e640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c16450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5535c640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 0x55fc81c15520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea40009610 tx=0x7fea400b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cac88e4d3ee8953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea397fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea397fa640 1 -- 172.21.15.31:0/3733353399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40020040 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea397fa640 1 -- 172.21.15.31:0/3733353399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fea4009a7a0 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea48036030 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.155+0000 7fea397fa640 1 -- 172.21.15.31:0/3733353399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea40005180 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 10 client.16257 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc81c82460 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea397fa640 1 -- 172.21.15.31:0/3733353399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48036030 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea397fa640 1 -- 172.21.15.31:0/3733353399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea400b4170 con 0x55fc81c06840 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 10 client.16257 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 20 client.16257 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c84de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 4 client.16257 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc81b71540 con 0x55fc81c1bf30 2024-04-25T00:41:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.156+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c84de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.157+0000 7fea397fa640 10 client.16257 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.157+0000 7fea55b5d640 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c84de0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e09241e9c3cc2f server_cookie=2a5799aa90f55984 in_seq=0 out_seq=0 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea397fa640 1 -- 172.21.15.31:0/3733353399 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc81b71540 con 0x55fc81c1bf30 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea397fa640 10 client.16257 handle_command_reply: tid=0 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 client.16257 shutdown 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 msgr2=0x55fc81c15520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 0x55fc81c15520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea40009610 tx=0x7fea400b8040 comp rx=0 tx=0).stop 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 shutdown_connections 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55fc81c1bf30 0x55fc81c84de0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc81c0b9b0 0x55fc81c16450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fc81c06840 0x55fc81c15520 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea397fa640 0 client.16257 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 --2- 172.21.15.31:0/3733353399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc81c05e50 0x55fc81c14fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 >> 172.21.15.31:0/3733353399 conn(0x55fc81c014c0 msgr2=0x55fc81c04240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 shutdown_connections 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 1 -- 172.21.15.31:0/3733353399 wait complete. 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.171+0000 7fea5aba9740 20 client.16257 trim_cache size 0 max 16384 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:41:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:41:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fea3c0a2c30 msgr2=0x7fea3c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fea3c0a2c30 0x7fea3c0a50f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea48031770 tx=0x7fea48049000 comp rx=0 tx=0).stop 2024-04-25T00:41:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 msgr2=0x7fea50073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 0x7fea50073b30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d770 tx=0x7fea4000dc30 comp rx=0 tx=0).stop 2024-04-25T00:41:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 shutdown_connections 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea50103380 0x7fea50074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fea3c0a2c30 0x7fea3c0a50f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea501029c0 0x7fea50073b30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 --2- 172.21.15.31:0/1104840342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea501017d0 0x7fea500754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 >> 172.21.15.31:0/1104840342 conn(0x7fea500fcd60 msgr2=0x7fea50105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 shutdown_connections 2024-04-25T00:41:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:14.172+0000 7fea57dc1640 1 -- 172.21.15.31:0/1104840342 wait complete. 2024-04-25T00:41:15.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:14 smithi031 ceph-mon[29167]: pgmap v681: 97 pgs: 97 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:15.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:14 smithi169 ceph-mon[33112]: pgmap v681: 97 pgs: 97 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:15.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:14 smithi040 ceph-mon[32945]: pgmap v681: 97 pgs: 97 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:17.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:16 smithi031 ceph-mon[29167]: pgmap v682: 97 pgs: 97 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:41:17.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:16 smithi169 ceph-mon[33112]: pgmap v682: 97 pgs: 97 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:41:17.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:16 smithi040 ceph-mon[32945]: pgmap v682: 97 pgs: 97 active+clean; 3.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:41:19.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:18 smithi031 ceph-mon[29167]: pgmap v683: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:41:19.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:18 smithi169 ceph-mon[33112]: pgmap v683: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:41:19.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:18 smithi040 ceph-mon[32945]: pgmap v683: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-04-25T00:41:21.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:20 smithi031 ceph-mon[29167]: pgmap v684: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:21.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:20 smithi169 ceph-mon[33112]: pgmap v684: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:21.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:20 smithi040 ceph-mon[32945]: pgmap v684: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:23.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:22 smithi031 ceph-mon[29167]: pgmap v685: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:23.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:22 smithi169 ceph-mon[33112]: pgmap v685: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:23.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:22 smithi040 ceph-mon[32945]: pgmap v685: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:25.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:24 smithi031 ceph-mon[29167]: pgmap v686: 97 pgs: 97 active+clean; 3.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:25.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:24 smithi169 ceph-mon[33112]: pgmap v686: 97 pgs: 97 active+clean; 3.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:25.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:24 smithi040 ceph-mon[32945]: pgmap v686: 97 pgs: 97 active+clean; 3.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:27.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:26 smithi031 ceph-mon[29167]: pgmap v687: 97 pgs: 97 active+clean; 3.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 -- 172.21.15.31:0/2886511989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e441017d0 msgr2=0x7f2e44101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2886511989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e441017d0 0x7f2e44101bf0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009900 tx=0x7f2e3402a410 comp rx=0 tx=0).stop 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 -- 172.21.15.31:0/2886511989 shutdown_connections 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2886511989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e44103380 0x7f2e44109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2886511989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e441029c0 0x7f2e44102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2886511989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e441017d0 0x7f2e44101bf0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 -- 172.21.15.31:0/2886511989 >> 172.21.15.31:0/2886511989 conn(0x7f2e440fcd60 msgr2=0x7f2e440ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 -- 172.21.15.31:0/2886511989 shutdown_connections 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.272+0000 7f2e48b70640 1 -- 172.21.15.31:0/2886511989 wait complete. 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.273+0000 7f2e48b70640 1 Processor -- start 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.273+0000 7f2e48b70640 1 -- start start 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e441017d0 0x7f2e440754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e441029c0 0x7f2e44073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 0x7f2e44074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410d340 con 0x7f2e44103380 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410d1c0 con 0x7f2e441029c0 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410d4c0 con 0x7f2e441017d0 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 0x7f2e44074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 0x7f2e44074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42768/0 (socket says 172.21.15.31:42768) 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 -- 172.21.15.31:0/2435442693 learned_addr learned my addr 172.21.15.31:0/2435442693 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e4259c640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e441017d0 0x7f2e440754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e441029c0 0x7f2e44073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 -- 172.21.15.31:0/2435442693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e441017d0 msgr2=0x7f2e440754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e441017d0 0x7f2e440754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 -- 172.21.15.31:0/2435442693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e441029c0 msgr2=0x7f2e44073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e441029c0 0x7f2e44073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 -- 172.21.15.31:0/2435442693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e44074900 con 0x7f2e44103380 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e42d9d640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 0x7f2e44074070 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00eaa0 tx=0x7f2e3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e337fe640 1 -- 172.21.15.31:0/2435442693 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00cdb0 con 0x7f2e44103380 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e337fe640 1 -- 172.21.15.31:0/2435442693 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c005020 con 0x7f2e44103380 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.274+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441a1a70 con 0x7f2e44103380 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.275+0000 7f2e337fe640 1 -- 172.21.15.31:0/2435442693 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0116d0 con 0x7f2e44103380 2024-04-25T00:41:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.275+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a1fc0 con 0x7f2e44103380 2024-04-25T00:41:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.277+0000 7f2e337fe640 1 -- 172.21.15.31:0/2435442693 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2e3c0051d0 con 0x7f2e44103380 2024-04-25T00:41:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.277+0000 7f2e4b522740 1 Processor -- start 2024-04-25T00:41:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.277+0000 7f2e4b522740 1 -- start start 2024-04-25T00:41:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed7fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4b522740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56171ecdbcb0 con 0x56171ed7d9b0 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4b522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56171ec845b0 con 0x56171ed78840 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4b522740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56171ecd7c70 con 0x56171ed77e50 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed7fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed7fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42780/0 (socket says 172.21.15.31:42780) 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 -- 172.21.15.31:0/2745640097 learned_addr learned my addr 172.21.15.31:0/2745640097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e337fe640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2e2c0a2cb0 0x7f2e2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e4259c640 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e42d9d640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2e2c0a2cb0 0x7f2e2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 -- 172.21.15.31:0/2745640097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 msgr2=0x56171ed78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 -- 172.21.15.31:0/2745640097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 msgr2=0x56171ed7d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed7d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 -- 172.21.15.31:0/2745640097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171e9f0210 con 0x56171ed7d9b0 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e337fe640 1 -- 172.21.15.31:0/2435442693 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2e3c0ceeb0 con 0x7f2e44103380 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.278+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed7fe40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2e380058c0 tx=0x7f2e3800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aac42a25c19dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e30ff9640 1 -- 172.21.15.31:0/2745640097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380142d0 con 0x56171ed7d9b0 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e42d9d640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2e2c0a2cb0 0x7f2e2c0a5170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c09b670 tx=0x7f2e3c0663e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e30ff9640 1 -- 172.21.15.31:0/2745640097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e380054d0 con 0x56171ed7d9b0 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e30ff9640 1 -- 172.21.15.31:0/2745640097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004e50 con 0x56171ed7d9b0 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 -- 172.21.15.31:0/2745640097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 msgr2=0x56171ed7fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed7fe40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2e380058c0 tx=0x7f2e3800c4d0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 -- 172.21.15.31:0/2745640097 shutdown_connections 2024-04-25T00:41:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed7fe40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 --2- 172.21.15.31:0/2745640097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.279+0000 7f2e4b522740 1 -- 172.21.15.31:0/2745640097 >> 172.21.15.31:0/2745640097 conn(0x56171ed734c0 msgr2=0x56171ed738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 -- 172.21.15.31:0/2745640097 shutdown_connections 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 -- 172.21.15.31:0/2745640097 wait complete. 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 -- start start 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 Processor -- start 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed82220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed82760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56171ed80960 con 0x56171ed7d9b0 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56171ec97b50 con 0x56171ed78840 2024-04-25T00:41:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56171ed80ae0 con 0x56171ed77e50 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4b522740 10 client.0 _ll_register_callbacks cb 0x56171ecd2ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e4259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed897a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42782/0 (socket says 172.21.15.31:42782) 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e41d9b640 1 -- 172.21.15.31:0/3846588723 learned_addr learned my addr 172.21.15.31:0/3846588723 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.280+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed82220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e41d9b640 1 -- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 msgr2=0x56171ed82220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed82220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e41d9b640 1 -- 172.21.15.31:0/3846588723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 msgr2=0x56171ed82760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed82760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e41d9b640 1 -- 172.21.15.31:0/3846588723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171eabc620 con 0x56171ed7d9b0 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed897a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005750 tx=0x7f2e38014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445c16d6624f6893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e29ffb640 1 -- 172.21.15.31:0/3846588723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004fe0 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e29ffb640 1 -- 172.21.15.31:0/3846588723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2e38005180 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e380054d0 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e29ffb640 1 -- 172.21.15.31:0/3846588723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e3800f760 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e4b522740 10 client.16269 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56171ed8cb10 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.281+0000 7f2e29ffb640 1 -- 172.21.15.31:0/3846588723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e380054d0 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e29ffb640 1 -- 172.21.15.31:0/3846588723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e380128a0 con 0x56171ed7d9b0 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4b522740 10 client.16269 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4b522740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4b522740 20 client.16269 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4b522740 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171ed8e120 0x56171ed90510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4b522740 4 client.16269 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56171ece91f0 con 0x56171ed8e120 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4259c640 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171ed8e120 0x56171ed90510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e4259c640 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171ed8e120 0x56171ed90510 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30645e57594ac0bb server_cookie=c91772052022e334 in_seq=0 out_seq=0 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.282+0000 7f2e29ffb640 10 client.16269 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e29ffb640 1 -- 172.21.15.31:0/3846588723 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56171ece91f0 con 0x56171ed8e120 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e29ffb640 10 client.16269 handle_command_reply: tid=0 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 client.16269 shutdown 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 msgr2=0x56171ed897a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed897a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005750 tx=0x7f2e38014f10 comp rx=0 tx=0).stop 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 shutdown_connections 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171ed8e120 0x56171ed90510 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed897a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed78840 0x56171ed82760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e29ffb640 0 client.16269 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 --2- 172.21.15.31:0/3846588723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed77e50 0x56171ed82220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 >> 172.21.15.31:0/3846588723 conn(0x56171ed734c0 msgr2=0x56171ed725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.283+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 shutdown_connections 2024-04-25T00:41:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.284+0000 7f2e4b522740 1 -- 172.21.15.31:0/3846588723 wait complete. 2024-04-25T00:41:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.284+0000 7f2e4b522740 20 client.16269 trim_cache size 0 max 16384 2024-04-25T00:41:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 Processor -- start 2024-04-25T00:41:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 -- start start 2024-04-25T00:41:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 0x56171ed7d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed7d9b0 0x56171ed72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441aae20 con 0x56171ed77e50 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ac960 con 0x56171ed7d9b0 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4b522740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ab4f0 con 0x56171ed78840 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 0x56171ed7d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 0x56171ed7d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42798/0 (socket says 172.21.15.31:42798) 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 -- 172.21.15.31:0/4057107783 learned_addr learned my addr 172.21.15.31:0/4057107783 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 -- 172.21.15.31:0/4057107783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 msgr2=0x56171ed89760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e4259c640 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed89760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed89760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 -- 172.21.15.31:0/4057107783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed7d9b0 msgr2=0x56171ed72440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed7d9b0 0x56171ed72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 -- 172.21.15.31:0/4057107783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171ed73f10 con 0x56171ed77e50 2024-04-25T00:41:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.299+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 0x56171ed7d270 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2e38012d10 tx=0x7f2e38005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6d96e161e662d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e2bfff640 1 -- 172.21.15.31:0/4057107783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38026030 con 0x56171ed77e50 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e2bfff640 1 -- 172.21.15.31:0/4057107783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e38022440 con 0x56171ed77e50 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e2bfff640 1 -- 172.21.15.31:0/4057107783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38034040 con 0x56171ed77e50 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 -- 172.21.15.31:0/4057107783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 msgr2=0x56171ed7d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 0x56171ed7d270 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2e38012d10 tx=0x7f2e38005fd0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 -- 172.21.15.31:0/4057107783 shutdown_connections 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed7d9b0 0x56171ed72440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 --2- 172.21.15.31:0/4057107783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed77e50 0x56171ed7d270 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 -- 172.21.15.31:0/4057107783 >> 172.21.15.31:0/4057107783 conn(0x56171ed734c0 msgr2=0x56171ed71d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 -- 172.21.15.31:0/4057107783 shutdown_connections 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.300+0000 7f2e4b522740 1 -- 172.21.15.31:0/4057107783 wait complete. 2024-04-25T00:41:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 -- start start 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 Processor -- start 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed77e50 0x56171ed868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed86e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed87e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ac290 con 0x56171ed7d9b0 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441abbc0 con 0x56171ed77e50 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441a8e00 con 0x56171ed78840 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4b522740 10 client.0 _ll_register_callbacks cb 0x7f2e3800cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed87e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed87e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42802/0 (socket says 172.21.15.31:42802) 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 -- 172.21.15.31:0/1044596365 learned_addr learned my addr 172.21.15.31:0/1044596365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed77e50 0x56171ed868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4259c640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed86e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 -- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 msgr2=0x56171ed86e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed86e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 -- 172.21.15.31:0/1044596365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed77e50 msgr2=0x56171ed868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed77e50 0x56171ed868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e41d9b640 1 -- 172.21.15.31:0/1044596365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171ed88420 con 0x56171ed7d9b0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.301+0000 7f2e4259c640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e41d9b640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed87e30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005d00 tx=0x7f2e38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e051e718c0f20ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e29ffb640 1 -- 172.21.15.31:0/1044596365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38026070 con 0x56171ed7d9b0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e29ffb640 1 -- 172.21.15.31:0/1044596365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2e3800fd90 con 0x56171ed7d9b0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e38022440 con 0x56171ed7d9b0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e29ffb640 1 -- 172.21.15.31:0/1044596365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e3800f990 con 0x56171ed7d9b0 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 10 client.16275 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56171ed89730 con 0x56171ed7d9b0 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e29ffb640 1 -- 172.21.15.31:0/1044596365 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e38022440 con 0x56171ed7d9b0 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e29ffb640 1 -- 172.21.15.31:0/1044596365 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e38012710 con 0x56171ed7d9b0 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 10 client.16275 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 20 client.16275 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171edf4a30 0x56171edf6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 4 client.16275 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.302+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56171ece3540 con 0x56171edf4a30 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.303+0000 7f2e4259c640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171edf4a30 0x56171edf6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.303+0000 7f2e4259c640 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171edf4a30 0x56171edf6e50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177d0763c4ee744 server_cookie=d296870d66a25ae7 in_seq=0 out_seq=0 2024-04-25T00:41:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.303+0000 7f2e29ffb640 10 client.16275 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e29ffb640 1 -- 172.21.15.31:0/1044596365 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56171ece3540 con 0x56171edf4a30 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e29ffb640 10 client.16275 handle_command_reply: tid=0 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 client.16275 shutdown 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 msgr2=0x56171ed87e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed87e30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005d00 tx=0x7f2e38005840 comp rx=0 tx=0).stop 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 shutdown_connections 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x56171edf4a30 0x56171edf6e50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56171ed7d9b0 0x56171ed87e30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171ed78840 0x56171ed86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e29ffb640 0 client.16275 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 --2- 172.21.15.31:0/1044596365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56171ed77e50 0x56171ed868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 >> 172.21.15.31:0/1044596365 conn(0x56171ed734c0 msgr2=0x56171ed71d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 shutdown_connections 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 1 -- 172.21.15.31:0/1044596365 wait complete. 2024-04-25T00:41:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.319+0000 7f2e4b522740 20 client.16275 trim_cache size 0 max 16384 2024-04-25T00:41:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:41:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:41:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:41:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:41:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2e2c0a2cb0 msgr2=0x7f2e2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2e2c0a2cb0 0x7f2e2c0a5170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c09b670 tx=0x7f2e3c0663e0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 msgr2=0x7f2e44074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 0x7f2e44074070 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00eaa0 tx=0x7f2e3c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 shutdown_connections 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2e2c0a2cb0 0x7f2e2c0a5170 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44103380 0x7f2e44074070 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e441029c0 0x7f2e44073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 --2- 172.21.15.31:0/2435442693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e441017d0 0x7f2e440754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 >> 172.21.15.31:0/2435442693 conn(0x7f2e440fcd60 msgr2=0x7f2e44105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 shutdown_connections 2024-04-25T00:41:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:27.320+0000 7f2e48b70640 1 -- 172.21.15.31:0/2435442693 wait complete. 2024-04-25T00:41:27.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:26 smithi169 ceph-mon[33112]: pgmap v687: 97 pgs: 97 active+clean; 3.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:27.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:26 smithi040 ceph-mon[32945]: pgmap v687: 97 pgs: 97 active+clean; 3.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:29.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:28 smithi031 ceph-mon[29167]: pgmap v688: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:41:29.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:28 smithi169 ceph-mon[33112]: pgmap v688: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:41:29.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:28 smithi040 ceph-mon[32945]: pgmap v688: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-04-25T00:41:31.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:30 smithi031 ceph-mon[29167]: pgmap v689: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T00:41:31.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:30 smithi169 ceph-mon[33112]: pgmap v689: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T00:41:31.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:30 smithi040 ceph-mon[32945]: pgmap v689: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T00:41:33.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:32 smithi031 ceph-mon[29167]: pgmap v690: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:41:33.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:32 smithi169 ceph-mon[33112]: pgmap v690: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:41:33.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:32 smithi040 ceph-mon[32945]: pgmap v690: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T00:41:35.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:34 smithi031 ceph-mon[29167]: pgmap v691: 97 pgs: 97 active+clean; 3.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:35.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:34 smithi169 ceph-mon[33112]: pgmap v691: 97 pgs: 97 active+clean; 3.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:35.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:34 smithi040 ceph-mon[32945]: pgmap v691: 97 pgs: 97 active+clean; 3.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-04-25T00:41:37.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:36 smithi031 ceph-mon[29167]: pgmap v692: 97 pgs: 97 active+clean; 3.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:41:37.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:36 smithi169 ceph-mon[33112]: pgmap v692: 97 pgs: 97 active+clean; 3.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:41:37.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:36 smithi040 ceph-mon[32945]: pgmap v692: 97 pgs: 97 active+clean; 3.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T00:41:39.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:38 smithi031 ceph-mon[29167]: pgmap v693: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:39.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:38 smithi169 ceph-mon[33112]: pgmap v693: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:39.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:38 smithi040 ceph-mon[32945]: pgmap v693: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 -- 172.21.15.31:0/265523229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef440ffa90 msgr2=0x7fef440ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 --2- 172.21.15.31:0/265523229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef440ffa90 0x7fef440ffeb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fef38009930 tx=0x7fef3802a420 comp rx=0 tx=0).stop 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 -- 172.21.15.31:0/265523229 shutdown_connections 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 --2- 172.21.15.31:0/265523229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef44103420 0x7fef44109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 --2- 172.21.15.31:0/265523229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef44102a60 0x7fef44102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 --2- 172.21.15.31:0/265523229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef440ffa90 0x7fef440ffeb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 -- 172.21.15.31:0/265523229 >> 172.21.15.31:0/265523229 conn(0x7fef440fb3c0 msgr2=0x7fef440fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 -- 172.21.15.31:0/265523229 shutdown_connections 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.419+0000 7fef4bb09640 1 -- 172.21.15.31:0/265523229 wait complete. 2024-04-25T00:41:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 Processor -- start 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 -- start start 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440ffa90 0x7fef44197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef44102a60 0x7fef44197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 0x7fef4419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410d310 con 0x7fef44103420 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410d190 con 0x7fef440ffa90 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4bb09640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410d490 con 0x7fef44102a60 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4a0a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 0x7fef4419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4a0a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 0x7fef4419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58636/0 (socket says 172.21.15.31:58636) 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef4a0a6640 1 -- 172.21.15.31:0/2573464292 learned_addr learned my addr 172.21.15.31:0/2573464292 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef490a4640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef44102a60 0x7fef44197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.420+0000 7fef498a5640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440ffa90 0x7fef44197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4a0a6640 1 -- 172.21.15.31:0/2573464292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef44102a60 msgr2=0x7fef44197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4a0a6640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef44102a60 0x7fef44197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4a0a6640 1 -- 172.21.15.31:0/2573464292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440ffa90 msgr2=0x7fef44197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4a0a6640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440ffa90 0x7fef44197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4a0a6640 1 -- 172.21.15.31:0/2573464292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4419f570 con 0x7fef44103420 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4a0a6640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 0x7fef4419ef00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00cce0 tx=0x7fef3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef36ffd640 1 -- 172.21.15.31:0/2573464292 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00f450 con 0x7fef44103420 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef36ffd640 1 -- 172.21.15.31:0/2573464292 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef3c00faa0 con 0x7fef44103420 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef36ffd640 1 -- 172.21.15.31:0/2573464292 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c018b70 con 0x7fef44103420 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4419f860 con 0x7fef44103420 2024-04-25T00:41:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.421+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4419fd60 con 0x7fef44103420 2024-04-25T00:41:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.423+0000 7fef36ffd640 1 -- 172.21.15.31:0/2573464292 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fef3c018d10 con 0x7fef44103420 2024-04-25T00:41:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 Processor -- start 2024-04-25T00:41:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 -- start start 2024-04-25T00:41:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f569680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569c50 0x55793f56e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f571250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55793f4c8cb0 con 0x55793f56edc0 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55793f4715b0 con 0x55793f569c50 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef4e80b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55793f4c4c70 con 0x55793f569260 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.424+0000 7fef36ffd640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fef300a2cb0 0x7fef300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f571250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f571250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58648/0 (socket says 172.21.15.31:58648) 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef498a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569c50 0x55793f56e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 -- 172.21.15.31:0/2990786217 learned_addr learned my addr 172.21.15.31:0/2990786217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f569680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef4a0a6640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fef300a2cb0 0x7fef300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef36ffd640 1 -- 172.21.15.31:0/2573464292 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fef3c0cf460 con 0x7fef44103420 2024-04-25T00:41:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 -- 172.21.15.31:0/2990786217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 msgr2=0x55793f569680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f569680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 -- 172.21.15.31:0/2990786217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569c50 msgr2=0x55793f56e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569c50 0x55793f56e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 -- 172.21.15.31:0/2990786217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793f1dd210 con 0x55793f56edc0 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef498a5640 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569c50 0x55793f56e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef490a4640 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f571250 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fef40012770 tx=0x7fef4000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbf6e61eed571dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef4a0a6640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fef300a2cb0 0x7fef300a5170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0cf810 tx=0x7fef3c00af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef2ffff640 1 -- 172.21.15.31:0/2990786217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40012cf0 con 0x55793f56edc0 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef2ffff640 1 -- 172.21.15.31:0/2990786217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40005020 con 0x55793f56edc0 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.425+0000 7fef2ffff640 1 -- 172.21.15.31:0/2990786217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef400049a0 con 0x55793f56edc0 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 -- 172.21.15.31:0/2990786217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 msgr2=0x55793f571250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f571250 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fef40012770 tx=0x7fef4000b7c0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 -- 172.21.15.31:0/2990786217 shutdown_connections 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f571250 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569c50 0x55793f56e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2990786217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f569680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 -- 172.21.15.31:0/2990786217 >> 172.21.15.31:0/2990786217 conn(0x55793f5604c0 msgr2=0x55793f5608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 -- 172.21.15.31:0/2990786217 shutdown_connections 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.426+0000 7fef4e80b740 1 -- 172.21.15.31:0/2990786217 wait complete. 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 -- start start 2024-04-25T00:41:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 Processor -- start 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f573650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 0x55793f573b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f56edc0 0x55793f57abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55793f571d70 con 0x55793f569c50 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55793f484b50 con 0x55793f56edc0 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55793f571ef0 con 0x55793f569260 2024-04-25T00:41:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef4e80b740 10 client.0 _ll_register_callbacks cb 0x55793f4bfca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef498a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 0x55793f573b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef498a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 0x55793f573b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58652/0 (socket says 172.21.15.31:58652) 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.427+0000 7fef498a5640 1 -- 172.21.15.31:0/214136762 learned_addr learned my addr 172.21.15.31:0/214136762 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef490a4640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f56edc0 0x55793f57abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef490a4640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f573650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef498a5640 1 -- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 msgr2=0x55793f573650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef498a5640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f573650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef498a5640 1 -- 172.21.15.31:0/214136762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f56edc0 msgr2=0x55793f57abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef498a5640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f56edc0 0x55793f57abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef498a5640 1 -- 172.21.15.31:0/214136762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793f2a9620 con 0x55793f569c50 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef490a4640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f573650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef498a5640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 0x55793f573b90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fef38000c00 tx=0x7fef38004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b3fb6fadb37f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef2d7fa640 1 -- 172.21.15.31:0/214136762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38035040 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef2d7fa640 1 -- 172.21.15.31:0/214136762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fef38005030 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef40005020 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef2d7fa640 1 -- 172.21.15.31:0/214136762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef38005230 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef4e80b740 10 client.16287 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55793f57dfb0 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.428+0000 7fef2d7fa640 1 -- 172.21.15.31:0/214136762 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40005020 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef2d7fa640 1 -- 172.21.15.31:0/214136762 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef3804e070 con 0x55793f569c50 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef4e80b740 10 client.16287 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef4e80b740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef4e80b740 20 client.16287 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef4e80b740 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f57f6f0 0x55793f581b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef4e80b740 4 client.16287 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55793f4d61f0 con 0x55793f57f6f0 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef490a4640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f57f6f0 0x55793f581b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef490a4640 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f57f6f0 0x55793f581b00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac89999e4964ffd7 server_cookie=1b18191d29d2bf2 in_seq=0 out_seq=0 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.429+0000 7fef2d7fa640 10 client.16287 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef2d7fa640 1 -- 172.21.15.31:0/214136762 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55793f4d61f0 con 0x55793f57f6f0 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef2d7fa640 10 client.16287 handle_command_reply: tid=0 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 client.16287 shutdown 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 msgr2=0x55793f573b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 0x55793f573b90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fef38000c00 tx=0x7fef38004060 comp rx=0 tx=0).stop 2024-04-25T00:41:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 shutdown_connections 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f57f6f0 0x55793f581b00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f56edc0 0x55793f57abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef2d7fa640 0 client.16287 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f569c50 0x55793f573b90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 --2- 172.21.15.31:0/214136762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569260 0x55793f573650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.430+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 >> 172.21.15.31:0/214136762 conn(0x55793f5604c0 msgr2=0x55793f55f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.431+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 shutdown_connections 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.431+0000 7fef4e80b740 1 -- 172.21.15.31:0/214136762 wait complete. 2024-04-25T00:41:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.431+0000 7fef4e80b740 20 client.16287 trim_cache size 0 max 16384 2024-04-25T00:41:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 Processor -- start 2024-04-25T00:41:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 -- start start 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f563240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f57ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f563780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fef441a6980 con 0x55793f56edc0 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef441a84c0 con 0x55793f569260 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef4e80b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef441a7050 con 0x55793f569c50 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f563780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f563780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58658/0 (socket says 172.21.15.31:58658) 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 -- 172.21.15.31:0/2624100378 learned_addr learned my addr 172.21.15.31:0/2624100378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef498a5640 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f57ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f563240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 -- 172.21.15.31:0/2624100378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 msgr2=0x55793f57ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f57ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 -- 172.21.15.31:0/2624100378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 msgr2=0x55793f563240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f563240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.446+0000 7fef490a4640 1 -- 172.21.15.31:0/2624100378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793f582180 con 0x55793f56edc0 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef490a4640 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f563780 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b800 tx=0x7fef40017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d6c0a39eaa1216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef2f7fe640 1 -- 172.21.15.31:0/2624100378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40005280 con 0x55793f56edc0 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef2f7fe640 1 -- 172.21.15.31:0/2624100378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40005420 con 0x55793f56edc0 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef2f7fe640 1 -- 172.21.15.31:0/2624100378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40010b30 con 0x55793f56edc0 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 -- 172.21.15.31:0/2624100378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 msgr2=0x55793f563780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f563780 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b800 tx=0x7fef40017040 comp rx=0 tx=0).stop 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 -- 172.21.15.31:0/2624100378 shutdown_connections 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f563780 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f57ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 --2- 172.21.15.31:0/2624100378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f563240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 -- 172.21.15.31:0/2624100378 >> 172.21.15.31:0/2624100378 conn(0x55793f5604c0 msgr2=0x55793f56b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 -- 172.21.15.31:0/2624100378 shutdown_connections 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.447+0000 7fef4e80b740 1 -- 172.21.15.31:0/2624100378 wait complete. 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 -- start start 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 Processor -- start 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f5782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f5787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f5797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fef441a7df0 con 0x55793f56edc0 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef441a7720 con 0x55793f569260 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef441a4960 con 0x55793f569c50 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef4e80b740 10 client.0 _ll_register_callbacks cb 0x7fef38031580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef490a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f5797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef490a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f5797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58668/0 (socket says 172.21.15.31:58668) 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef490a4640 1 -- 172.21.15.31:0/695747154 learned_addr learned my addr 172.21.15.31:0/695747154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.448+0000 7fef490a4640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f5782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef498a5640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f5787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef490a4640 1 -- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 msgr2=0x55793f5787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef490a4640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f5787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef490a4640 1 -- 172.21.15.31:0/695747154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 msgr2=0x55793f5782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef490a4640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f5782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef490a4640 1 -- 172.21.15.31:0/695747154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793f579de0 con 0x55793f56edc0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef498a5640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f5787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef490a4640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f5797f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef4000bbb0 tx=0x7fef4000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc680fb6365c349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef2d7fa640 1 -- 172.21.15.31:0/695747154 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4002a720 con 0x55793f56edc0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef2d7fa640 1 -- 172.21.15.31:0/695747154 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fef400108b0 con 0x55793f56edc0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef40005420 con 0x55793f56edc0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef2d7fa640 1 -- 172.21.15.31:0/695747154 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef40005260 con 0x55793f56edc0 2024-04-25T00:41:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef4e80b740 10 client.16293 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55793f5e5c80 con 0x55793f56edc0 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.449+0000 7fef2d7fa640 1 -- 172.21.15.31:0/695747154 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40005420 con 0x55793f56edc0 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef2d7fa640 1 -- 172.21.15.31:0/695747154 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef40032020 con 0x55793f56edc0 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef4e80b740 10 client.16293 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef4e80b740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef4e80b740 20 client.16293 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef4e80b740 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f5e6330 0x55793f5e8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef4e80b740 4 client.16293 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55793f4d0540 con 0x55793f5e6330 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.450+0000 7fef498a5640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f5e6330 0x55793f5e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.451+0000 7fef2d7fa640 10 client.16293 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.451+0000 7fef498a5640 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f5e6330 0x55793f5e8750 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e22a2b02f452226 server_cookie=159c1f5a232c75d5 in_seq=0 out_seq=0 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef2d7fa640 1 -- 172.21.15.31:0/695747154 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55793f4d0540 con 0x55793f5e6330 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef2d7fa640 10 client.16293 handle_command_reply: tid=0 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 client.16293 shutdown 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 msgr2=0x55793f5797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f5797f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef4000bbb0 tx=0x7fef4000b5b0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 shutdown_connections 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55793f5e6330 0x55793f5e8750 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55793f56edc0 0x55793f5797f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55793f569c50 0x55793f5787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef2d7fa640 0 client.16293 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 --2- 172.21.15.31:0/695747154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55793f569260 0x55793f5782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 >> 172.21.15.31:0/695747154 conn(0x55793f5604c0 msgr2=0x55793f55ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.467+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 shutdown_connections 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4e80b740 1 -- 172.21.15.31:0/695747154 wait complete. 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4e80b740 20 client.16293 trim_cache size 0 max 16384 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:41:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:41:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fef300a2cb0 msgr2=0x7fef300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fef300a2cb0 0x7fef300a5170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0cf810 tx=0x7fef3c00af30 comp rx=0 tx=0).stop 2024-04-25T00:41:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 msgr2=0x7fef4419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 0x7fef4419ef00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00cce0 tx=0x7fef3c00a620 comp rx=0 tx=0).stop 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 shutdown_connections 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fef300a2cb0 0x7fef300a5170 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef44103420 0x7fef4419ef00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef44102a60 0x7fef44197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 --2- 172.21.15.31:0/2573464292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440ffa90 0x7fef44197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 >> 172.21.15.31:0/2573464292 conn(0x7fef440fb3c0 msgr2=0x7fef44105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 shutdown_connections 2024-04-25T00:41:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:40.468+0000 7fef4bb09640 1 -- 172.21.15.31:0/2573464292 wait complete. 2024-04-25T00:41:41.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:40 smithi031 ceph-mon[29167]: pgmap v694: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:41.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:40 smithi169 ceph-mon[33112]: pgmap v694: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:41.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:40 smithi040 ceph-mon[32945]: pgmap v694: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:43.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:42 smithi031 ceph-mon[29167]: pgmap v695: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:41:43.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:42 smithi169 ceph-mon[33112]: pgmap v695: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:41:43.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:42 smithi040 ceph-mon[32945]: pgmap v695: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T00:41:45.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:44 smithi031 ceph-mon[29167]: pgmap v696: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:41:45.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:44 smithi169 ceph-mon[33112]: pgmap v696: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:41:45.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:44 smithi040 ceph-mon[32945]: pgmap v696: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-04-25T00:41:47.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:46 smithi031 ceph-mon[29167]: pgmap v697: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:41:47.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:46 smithi169 ceph-mon[33112]: pgmap v697: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:41:47.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:46 smithi040 ceph-mon[32945]: pgmap v697: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-25T00:41:49.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:48 smithi031 ceph-mon[29167]: pgmap v698: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:49.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:48 smithi169 ceph-mon[33112]: pgmap v698: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:49.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:48 smithi040 ceph-mon[32945]: pgmap v698: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-25T00:41:51.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:50 smithi031 ceph-mon[29167]: pgmap v699: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:51.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:50 smithi169 ceph-mon[33112]: pgmap v699: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:51.397 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:50 smithi040 ceph-mon[32945]: pgmap v699: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T00:41:53.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:52 smithi031 ceph-mon[29167]: pgmap v700: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:53.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:52 smithi169 ceph-mon[33112]: pgmap v700: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:53.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:52 smithi040 ceph-mon[32945]: pgmap v700: 97 pgs: 97 active+clean; 3.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T00:41:53.435 INFO:tasks.workunit.client.0.smithi031.stdout:561/600: open fd = 3 2024-04-25T00:41:53.435 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.435 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.435 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.435 INFO:tasks.workunit.client.0.smithi031.stdout:562/600: open fd = 3 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:563/600: open fd = 3 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:564/600: open fd = 3 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:565/600: open fd = 3 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:566/600: open fd = 3 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:567/600: open fd = 3 2024-04-25T00:41:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:568/600: open fd = 3 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:569/600: open fd = 3 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.438 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:570/600: open fd = 3 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:571/600: open fd = 3 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:572/600: open fd = 3 2024-04-25T00:41:53.439 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:573/600: open fd = 3 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:574/600: open fd = 3 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.440 INFO:tasks.workunit.client.0.smithi031.stdout:575/600: open fd = 3 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:576/600: open fd = 3 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:577/600: open fd = 3 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.441 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:578/600: open fd = 3 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:579/600: open fd = 3 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:580/600: open fd = 3 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:581/600: open fd = 3 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:582/600: open fd = 3 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:583/600: open fd = 3 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:584/600: open fd = 3 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:585/600: open fd = 3 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:586/600: open fd = 3 2024-04-25T00:41:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:587/600: open fd = 3 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:588/600: open fd = 3 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:589/600: open fd = 3 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:590/600: open fd = 3 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:591/600: open fd = 3 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:592/600: open fd = 3 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.446 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:593/600: open fd = 3 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:594/600: open fd = 3 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:595/600: open fd = 3 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:596/600: open fd = 3 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:597/600: open fd = 3 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:598/600: open fd = 3 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.448 INFO:tasks.workunit.client.0.smithi031.stdout:599/600: open fd = 3 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:600/600: open fd = 3 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:write ret = 32 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:pread ret = 64 2024-04-25T00:41:53.449 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo OK 2024-04-25T00:41:53.450 INFO:tasks.workunit.client.0.smithi031.stdout:OK 2024-04-25T00:41:53.451 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T00:41:53.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:41:53.541 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-25T00:41:53.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 -- 172.21.15.31:0/459453681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac1074e0 msgr2=0x7ff3ac1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/459453681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac1074e0 0x7ff3ac1098f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009ca0 tx=0x7ff3a402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 -- 172.21.15.31:0/459453681 shutdown_connections 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/459453681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac1074e0 0x7ff3ac1098f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/459453681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/459453681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 0x7ff3ac104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 -- 172.21.15.31:0/459453681 >> 172.21.15.31:0/459453681 conn(0x7ff3ac0fbc50 msgr2=0x7ff3ac0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 -- 172.21.15.31:0/459453681 shutdown_connections 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 -- 172.21.15.31:0/459453681 wait complete. 2024-04-25T00:41:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.570+0000 7ff3b33ee640 1 Processor -- start 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 -- start start 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 0x7ff3ac197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac1074e0 0x7ff3ac19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d310 con 0x7ff3ac104bb0 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d190 con 0x7ff3ac102280 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b33ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10d490 con 0x7ff3ac1074e0 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36838/0 (socket says 172.21.15.31:36838) 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 -- 172.21.15.31:0/2431095558 learned_addr learned my addr 172.21.15.31:0/2431095558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b198b640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac1074e0 0x7ff3ac19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b118a640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 0x7ff3ac197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 -- 172.21.15.31:0/2431095558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac1074e0 msgr2=0x7ff3ac19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac1074e0 0x7ff3ac19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 -- 172.21.15.31:0/2431095558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 msgr2=0x7ff3ac197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 0x7ff3ac197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b0989640 1 -- 172.21.15.31:0/2431095558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac19f500 con 0x7ff3ac104bb0 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.571+0000 7ff3b118a640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 0x7ff3ac197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.572+0000 7ff3b0989640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac197e70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00d740 tx=0x7ff39c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.572+0000 7ff3a27fc640 1 -- 172.21.15.31:0/2431095558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c017070 con 0x7ff3ac104bb0 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.572+0000 7ff3a27fc640 1 -- 172.21.15.31:0/2431095558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff39c00de50 con 0x7ff3ac104bb0 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.572+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac19f7f0 con 0x7ff3ac104bb0 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.572+0000 7ff3a27fc640 1 -- 172.21.15.31:0/2431095558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c004d60 con 0x7ff3ac104bb0 2024-04-25T00:41:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.572+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac19fd40 con 0x7ff3ac104bb0 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3a27fc640 1 -- 172.21.15.31:0/2431095558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff39c004f00 con 0x7ff3ac104bb0 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 Processor -- start 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 -- start start 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02dee8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02deed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02deefe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c02de4bcb0 con 0x55c02dee8840 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c02ddf45b0 con 0x55c02deed9b0 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.575+0000 7ff3b61da740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c02de47c70 con 0x55c02dee7e50 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3a27fc640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3980a2cb0 0x7ff3980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02deed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02deed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36842/0 (socket says 172.21.15.31:36842) 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 -- 172.21.15.31:0/1240159999 learned_addr learned my addr 172.21.15.31:0/1240159999 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b198b640 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02dee8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3a27fc640 1 -- 172.21.15.31:0/2431095558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff39c0ce990 con 0x7ff3ac104bb0 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b0989640 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02deefe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b198b640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3980a2cb0 0x7ff3980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 -- 172.21.15.31:0/1240159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 msgr2=0x55c02dee8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02dee8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 -- 172.21.15.31:0/1240159999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 msgr2=0x55c02deefe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02deefe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 -- 172.21.15.31:0/1240159999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c02db60210 con 0x55c02dee8840 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b0989640 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02deefe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b118a640 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02deed340 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800bd40 tx=0x7ff3a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fc6f13a9cf6b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3977fe640 1 -- 172.21.15.31:0/1240159999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8015070 con 0x55c02dee8840 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.576+0000 7ff3b198b640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3980a2cb0 0x7ff3980a5170 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4004af0 tx=0x7ff3a4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3977fe640 1 -- 172.21.15.31:0/1240159999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a800be30 con 0x55c02dee8840 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3977fe640 1 -- 172.21.15.31:0/1240159999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8011960 con 0x55c02dee8840 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 -- 172.21.15.31:0/1240159999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 msgr2=0x55c02deed340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02deed340 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800bd40 tx=0x7ff3a800f730 comp rx=0 tx=0).stop 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 -- 172.21.15.31:0/1240159999 shutdown_connections 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02deefe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02deed340 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 --2- 172.21.15.31:0/1240159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02dee8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 -- 172.21.15.31:0/1240159999 >> 172.21.15.31:0/1240159999 conn(0x55c02dee34c0 msgr2=0x55c02dee38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 -- 172.21.15.31:0/1240159999 shutdown_connections 2024-04-25T00:41:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.577+0000 7ff3b61da740 1 -- 172.21.15.31:0/1240159999 wait complete. 2024-04-25T00:41:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 -- start start 2024-04-25T00:41:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 Processor -- start 2024-04-25T00:41:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02def22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02def2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02def9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c02def0960 con 0x55c02dee8840 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c02de07b50 con 0x55c02deed9b0 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c02def0ae0 con 0x55c02dee7e50 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b61da740 10 client.0 _ll_register_callbacks cb 0x55c02de42ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b0989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02def2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b0989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02def2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36850/0 (socket says 172.21.15.31:36850) 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.578+0000 7ff3b0989640 1 -- 172.21.15.31:0/3902706196 learned_addr learned my addr 172.21.15.31:0/3902706196 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b198b640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02def9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b118a640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02def22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b0989640 1 -- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 msgr2=0x55c02def22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b0989640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02def22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b0989640 1 -- 172.21.15.31:0/3902706196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 msgr2=0x55c02def9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b0989640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02def9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b0989640 1 -- 172.21.15.31:0/3902706196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02dc2c620 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b0989640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02def2820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff39c097620 tx=0x7ff39c009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e4c6ba71d4901f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff394ff9640 1 -- 172.21.15.31:0/3902706196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c020040 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff394ff9640 1 -- 172.21.15.31:0/3902706196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff39c097830 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a800be30 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff394ff9640 1 -- 172.21.15.31:0/3902706196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff39c0b41c0 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b61da740 10 client.16305 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.579+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c02defcc00 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff394ff9640 1 -- 172.21.15.31:0/3902706196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a800be30 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff394ff9640 1 -- 172.21.15.31:0/3902706196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c09f120 con 0x55c02dee8840 2024-04-25T00:41:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b61da740 10 client.16305 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b61da740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b61da740 20 client.16305 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b61da740 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02defe390 0x55c02df007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b61da740 4 client.16305 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c02de591f0 con 0x55c02defe390 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.580+0000 7ff3b118a640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02defe390 0x55c02df007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff3b118a640 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02defe390 0x55c02df007a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84029c5f7f0259a server_cookie=606cfb9aa4427550 in_seq=0 out_seq=0 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff394ff9640 10 client.16305 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff394ff9640 1 -- 172.21.15.31:0/3902706196 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c02de591f0 con 0x55c02defe390 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff394ff9640 10 client.16305 handle_command_reply: tid=0 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff3b61da740 1 client.16305 shutdown 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 msgr2=0x55c02def2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.581+0000 7ff3b61da740 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02def2820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff39c097620 tx=0x7ff39c009e00 comp rx=0 tx=0).stop 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 shutdown_connections 2024-04-25T00:41:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02defe390 0x55c02df007a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02def9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02def2820 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff394ff9640 0 client.16305 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 --2- 172.21.15.31:0/3902706196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02def22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 >> 172.21.15.31:0/3902706196 conn(0x55c02dee34c0 msgr2=0x55c02dee25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 shutdown_connections 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 1 -- 172.21.15.31:0/3902706196 wait complete. 2024-04-25T00:41:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.582+0000 7ff3b61da740 20 client.16305 trim_cache size 0 max 16384 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 Processor -- start 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 -- start start 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 0x55c02dee6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee8840 0x55c02df050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1a8b90 con 0x55c02dee7e50 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1aa6d0 con 0x55c02deed9b0 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b61da740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1a9260 con 0x55c02dee8840 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b118a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 0x55c02dee6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b118a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 0x55c02dee6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36856/0 (socket says 172.21.15.31:36856) 2024-04-25T00:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b118a640 1 -- 172.21.15.31:0/2849823905 learned_addr learned my addr 172.21.15.31:0/2849823905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.597+0000 7ff3b198b640 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b0989640 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee8840 0x55c02df050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b118a640 1 -- 172.21.15.31:0/2849823905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee8840 msgr2=0x55c02df050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b118a640 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee8840 0x55c02df050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b118a640 1 -- 172.21.15.31:0/2849823905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 msgr2=0x55c02df055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b118a640 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b118a640 1 -- 172.21.15.31:0/2849823905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c02dee4230 con 0x55c02dee7e50 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b198b640 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff3b118a640 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 0x55c02dee6b80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801afd0 tx=0x7ff3a800b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced5d064b3750f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff396ffd640 1 -- 172.21.15.31:0/2849823905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8015040 con 0x55c02dee7e50 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff396ffd640 1 -- 172.21.15.31:0/2849823905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a80054f0 con 0x55c02dee7e50 2024-04-25T00:41:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.598+0000 7ff396ffd640 1 -- 172.21.15.31:0/2849823905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8004970 con 0x55c02dee7e50 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 -- 172.21.15.31:0/2849823905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 msgr2=0x55c02dee6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 0x55c02dee6b80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801afd0 tx=0x7ff3a800b660 comp rx=0 tx=0).stop 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 -- 172.21.15.31:0/2849823905 shutdown_connections 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee8840 0x55c02df050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 --2- 172.21.15.31:0/2849823905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee7e50 0x55c02dee6b80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 -- 172.21.15.31:0/2849823905 >> 172.21.15.31:0/2849823905 conn(0x55c02dee34c0 msgr2=0x55c02dee6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 -- 172.21.15.31:0/2849823905 shutdown_connections 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 -- 172.21.15.31:0/2849823905 wait complete. 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 -- start start 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.599+0000 7ff3b61da740 1 Processor -- start 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02df5fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02df60530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df67570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1aa000 con 0x55c02dee8840 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1a9930 con 0x55c02deed9b0 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1a6b70 con 0x55c02dee7e50 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b61da740 10 client.0 _ll_register_callbacks cb 0x7ff39c097460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02df60530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02df60530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36862/0 (socket says 172.21.15.31:36862) 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 -- 172.21.15.31:0/4014439140 learned_addr learned my addr 172.21.15.31:0/4014439140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b118a640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02df5fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 -- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 msgr2=0x55c02df5fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b198b640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df67570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02df5fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 -- 172.21.15.31:0/4014439140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 msgr2=0x55c02df67570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df67570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 -- 172.21.15.31:0/4014439140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02df67b60 con 0x55c02dee8840 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff3b0989640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02df60530 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009610 tx=0x7ff39c0bb2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd636eaec064ad1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.600+0000 7ff394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff394ff9640 1 -- 172.21.15.31:0/4014439140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c020040 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff394ff9640 1 -- 172.21.15.31:0/4014439140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff39c0b9e00 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a80054f0 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff394ff9640 1 -- 172.21.15.31:0/4014439140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff39c09af90 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 10 client.16311 fetch_fsmap learned FSMap version 26 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c02df68ee0 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff394ff9640 1 -- 172.21.15.31:0/4014439140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a80054f0 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff394ff9640 1 -- 172.21.15.31:0/4014439140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c09a080 con 0x55c02dee8840 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 10 client.16311 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 20 client.16311 populate_metadata read hostname 'smithi031' 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02df69930 0x55c02df6bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 4 client.16311 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.601+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 --> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c02de53540 con 0x55c02df69930 2024-04-25T00:41:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.602+0000 7ff3b118a640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02df69930 0x55c02df6bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.602+0000 7ff3b118a640 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02df69930 0x55c02df6bd50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f834b89ebb5498 server_cookie=fb85dec935b756e2 in_seq=0 out_seq=0 2024-04-25T00:41:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.602+0000 7ff394ff9640 10 client.16311 ms_handle_connect on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff394ff9640 1 -- 172.21.15.31:0/4014439140 <== mds.0 v2:172.21.15.169:6838/3109017383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c02de53540 con 0x55c02df69930 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff394ff9640 10 client.16311 handle_command_reply: tid=0 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 client.16311 shutdown 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 msgr2=0x55c02df60530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02df60530 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009610 tx=0x7ff39c0bb2c0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 shutdown_connections 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:6838/3109017383,v1:172.21.15.169:6839/3109017383] conn(0x55c02df69930 0x55c02df6bd50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c02deed9b0 0x55c02df67570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff394ff9640 0 client.16311 ms_handle_reset on v2:172.21.15.169:6838/3109017383 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c02dee8840 0x55c02df60530 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 --2- 172.21.15.31:0/4014439140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c02dee7e50 0x55c02df5fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 >> 172.21.15.31:0/4014439140 conn(0x55c02dee34c0 msgr2=0x55c02dee1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 shutdown_connections 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 1 -- 172.21.15.31:0/4014439140 wait complete. 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.622+0000 7ff3b61da740 20 client.16311 trim_cache size 0 max 16384 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T00:41:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T00:41:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3980a2cb0 msgr2=0x7ff3980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3980a2cb0 0x7ff3980a5170 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4004af0 tx=0x7ff3a4005d80 comp rx=0 tx=0).stop 2024-04-25T00:41:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 msgr2=0x7ff3ac197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac197e70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00d740 tx=0x7ff39c00dc00 comp rx=0 tx=0).stop 2024-04-25T00:41:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 shutdown_connections 2024-04-25T00:41:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac1074e0 0x7ff3ac19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3980a2cb0 0x7ff3980a5170 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac104bb0 0x7ff3ac197e70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 --2- 172.21.15.31:0/2431095558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac102280 0x7ff3ac197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 >> 172.21.15.31:0/2431095558 conn(0x7ff3ac0fbc50 msgr2=0x7ff3ac102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 shutdown_connections 2024-04-25T00:41:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T00:41:53.623+0000 7ff3b33ee640 1 -- 172.21.15.31:0/2431095558 wait complete. 2024-04-25T00:41:53.850 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:53.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:41:53.922 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:41:53.923 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T00:41:53.938 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:41:53.970 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T00:41:53.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 -- 172.21.15.31:0/2902592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2141029a0 msgr2=0x7fa214102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/2902592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2141029a0 0x7fa214102e20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa204009900 tx=0x7fa20402a410 comp rx=0 tx=0).stop 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 -- 172.21.15.31:0/2902592814 shutdown_connections 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/2902592814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa214103360 0x7fa214109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/2902592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2141029a0 0x7fa214102e20 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/2902592814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2141017b0 0x7fa214101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 -- 172.21.15.31:0/2902592814 >> 172.21.15.31:0/2902592814 conn(0x7fa2140fcd40 msgr2=0x7fa2140ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 -- 172.21.15.31:0/2902592814 shutdown_connections 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 -- 172.21.15.31:0/2902592814 wait complete. 2024-04-25T00:41:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 Processor -- start 2024-04-25T00:41:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.112+0000 7fa21a4f7640 1 -- start start 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa21a4f7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2141017b0 0x7fa214197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa21a4f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2141029a0 0x7fa214197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa21a4f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 0x7fa21419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa21a4f7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa21410d2e0 con 0x7fa214103360 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa21a4f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa21410d160 con 0x7fa2141029a0 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa21a4f7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa21410d460 con 0x7fa2141017b0 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 0x7fa21419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 0x7fa21419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36890/0 (socket says 172.21.15.31:36890) 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 -- 172.21.15.31:0/1862737912 learned_addr learned my addr 172.21.15.31:0/1862737912 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa213fff640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2141017b0 0x7fa214197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 -- 172.21.15.31:0/1862737912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2141017b0 msgr2=0x7fa214197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2141017b0 0x7fa214197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 -- 172.21.15.31:0/1862737912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2141029a0 msgr2=0x7fa214197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2141029a0 0x7fa214197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 -- 172.21.15.31:0/1862737912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21419f550 con 0x7fa214103360 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa213fff640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2141017b0 0x7fa214197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa218a94640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 0x7fa21419eee0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00eaa0 tx=0x7fa20c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.113+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00cdb0 con 0x7fa214103360 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.114+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21419f840 con 0x7fa214103360 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.114+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa20c005020 con 0x7fa214103360 2024-04-25T00:41:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.114+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0116d0 con 0x7fa214103360 2024-04-25T00:41:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.114+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21419fd90 con 0x7fa214103360 2024-04-25T00:41:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.114+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa214108ad0 con 0x7fa214103360 2024-04-25T00:41:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.116+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa20c0051d0 con 0x7fa214103360 2024-04-25T00:41:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.116+0000 7fa2117fa640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2000a2b90 0x7fa2000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.116+0000 7fa213fff640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2000a2b90 0x7fa2000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.116+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa20c0cf2b0 con 0x7fa214103360 2024-04-25T00:41:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.117+0000 7fa213fff640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2000a2b90 0x7fa2000a5050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa214102800 tx=0x7fa1fc00a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.121+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa20c016030 con 0x7fa214103360 2024-04-25T00:41:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.307+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa214062340 con 0x7fa214103360 2024-04-25T00:41:54.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.308+0000 7fa2117fa640 1 -- 172.21.15.31:0/1862737912 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7fa20c09c1e0 con 0x7fa214103360 2024-04-25T00:41:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:41:54.309 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.310+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2000a2b90 msgr2=0x7fa2000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.310+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2000a2b90 0x7fa2000a5050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa214102800 tx=0x7fa1fc00a5e0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.310+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 msgr2=0x7fa21419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.310+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 0x7fa21419eee0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00eaa0 tx=0x7fa20c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 shutdown_connections 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fa2000a2b90 0x7fa2000a5050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214103360 0x7fa21419eee0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2141029a0 0x7fa214197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 --2- 172.21.15.31:0/1862737912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2141017b0 0x7fa214197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 >> 172.21.15.31:0/1862737912 conn(0x7fa2140fcd40 msgr2=0x7fa214105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 shutdown_connections 2024-04-25T00:41:54.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.311+0000 7fa21a4f7640 1 -- 172.21.15.31:0/1862737912 wait complete. 2024-04-25T00:41:54.323 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:41:54.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:41:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 -- 172.21.15.31:0/4214550575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 msgr2=0x7f1320101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 --2- 172.21.15.31:0/4214550575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320101bc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1310009900 tx=0x7f131002a410 comp rx=0 tx=0).stop 2024-04-25T00:41:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 -- 172.21.15.31:0/4214550575 shutdown_connections 2024-04-25T00:41:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 --2- 172.21.15.31:0/4214550575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1320103350 0x7f1320109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 --2- 172.21.15.31:0/4214550575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1320102990 0x7f1320102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 --2- 172.21.15.31:0/4214550575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320101bc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 -- 172.21.15.31:0/4214550575 >> 172.21.15.31:0/4214550575 conn(0x7f13200fcd10 msgr2=0x7f13200ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 -- 172.21.15.31:0/4214550575 shutdown_connections 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.428+0000 7f1325410640 1 -- 172.21.15.31:0/4214550575 wait complete. 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 Processor -- start 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 -- start start 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1320102990 0x7f1320197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1320103350 0x7f132019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f132010d2d0 con 0x7f13201017a0 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f132010d150 con 0x7f1320103350 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1325410640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f132010d450 con 0x7f1320102990 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f131effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f131effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36896/0 (socket says 172.21.15.31:36896) 2024-04-25T00:41:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f131effd640 1 -- 172.21.15.31:0/3833496132 learned_addr learned my addr 172.21.15.31:0/3833496132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f131f7fe640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1320103350 0x7f132019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.429+0000 7f1317fff640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1320102990 0x7f1320197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131effd640 1 -- 172.21.15.31:0/3833496132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1320102990 msgr2=0x7f1320197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131effd640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1320102990 0x7f1320197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131effd640 1 -- 172.21.15.31:0/3833496132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1320103350 msgr2=0x7f132019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131effd640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1320103350 0x7f132019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131effd640 1 -- 172.21.15.31:0/3833496132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132019f630 con 0x7f13201017a0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131effd640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320197a10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f13100314e0 tx=0x7f1310031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131003f030 con 0x7f13201017a0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13100044b0 con 0x7f13201017a0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132019f8c0 con 0x7f13201017a0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310035070 con 0x7f13201017a0 2024-04-25T00:41:54.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.430+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132019fdb0 con 0x7f13201017a0 2024-04-25T00:41:54.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.431+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1320108be0 con 0x7f13201017a0 2024-04-25T00:41:54.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.432+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1310042030 con 0x7f13201017a0 2024-04-25T00:41:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.433+0000 7f131cff9640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f130c0a2b90 0x7f130c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.433+0000 7f1317fff640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f130c0a2b90 0x7f130c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.433+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f13100ef9a0 con 0x7f13201017a0 2024-04-25T00:41:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.433+0000 7f1317fff640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f130c0a2b90 0x7f130c0a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1320198f20 tx=0x7f1308005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.436+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f13100bc220 con 0x7f13201017a0 2024-04-25T00:41:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.622+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f132002d3c0 con 0x7f13201017a0 2024-04-25T00:41:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.622+0000 7f131cff9640 1 -- 172.21.15.31:0/3833496132 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f13100c0520 con 0x7f13201017a0 2024-04-25T00:41:54.623 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f130c0a2b90 msgr2=0x7f130c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f130c0a2b90 0x7f130c0a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1320198f20 tx=0x7f1308005f90 comp rx=0 tx=0).stop 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 msgr2=0x7f1320197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320197a10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f13100314e0 tx=0x7f1310031a30 comp rx=0 tx=0).stop 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 shutdown_connections 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1320103350 0x7f132019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1320102990 0x7f1320197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f130c0a2b90 0x7f130c0a5050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 --2- 172.21.15.31:0/3833496132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13201017a0 0x7f1320197a10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 >> 172.21.15.31:0/3833496132 conn(0x7f13200fcd10 msgr2=0x7f1320105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 shutdown_connections 2024-04-25T00:41:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.624+0000 7f1325410640 1 -- 172.21.15.31:0/3833496132 wait complete. 2024-04-25T00:41:54.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 -- 172.21.15.31:0/486771562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410102980 msgr2=0x7f2410102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 --2- 172.21.15.31:0/486771562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410102980 0x7f2410102e00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2404009900 tx=0x7f240402a410 comp rx=0 tx=0).stop 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 -- 172.21.15.31:0/486771562 shutdown_connections 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 --2- 172.21.15.31:0/486771562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2410103340 0x7f2410109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 --2- 172.21.15.31:0/486771562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410102980 0x7f2410102e00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 --2- 172.21.15.31:0/486771562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2410101790 0x7f2410101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 -- 172.21.15.31:0/486771562 >> 172.21.15.31:0/486771562 conn(0x7f24100fcd20 msgr2=0x7f24100ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 -- 172.21.15.31:0/486771562 shutdown_connections 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.741+0000 7f2415b77640 1 -- 172.21.15.31:0/486771562 wait complete. 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 Processor -- start 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 -- start start 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2410101790 0x7f24101979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2410102980 0x7f2410197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 0x7f241019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f241010d2a0 con 0x7f2410103340 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f241010d120 con 0x7f2410102980 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f2415b77640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f241010d420 con 0x7f2410101790 2024-04-25T00:41:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f240ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 0x7f241019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f240ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 0x7f241019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36926/0 (socket says 172.21.15.31:36926) 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f240ffff640 1 -- 172.21.15.31:0/396821209 learned_addr learned my addr 172.21.15.31:0/396821209 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f240effd640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2410102980 0x7f2410197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.742+0000 7f240f7fe640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2410101790 0x7f24101979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240ffff640 1 -- 172.21.15.31:0/396821209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2410101790 msgr2=0x7f24101979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240ffff640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2410101790 0x7f24101979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240ffff640 1 -- 172.21.15.31:0/396821209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2410102980 msgr2=0x7f2410197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240ffff640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2410102980 0x7f2410197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240ffff640 1 -- 172.21.15.31:0/396821209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019f5c0 con 0x7f2410103340 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240ffff640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 0x7f241019ef50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f240800d950 tx=0x7f240800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408017030 con 0x7f2410103340 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2408004020 con 0x7f2410103340 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019f8b0 con 0x7f2410103340 2024-04-25T00:41:54.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408003b20 con 0x7f2410103340 2024-04-25T00:41:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.743+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241019fd80 con 0x7f2410103340 2024-04-25T00:41:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.744+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410108b30 con 0x7f2410103340 2024-04-25T00:41:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.745+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f240801a070 con 0x7f2410103340 2024-04-25T00:41:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.746+0000 7f240cff9640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23f80a2b90 0x7f23f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.746+0000 7f240f7fe640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23f80a2b90 0x7f23f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.746+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f24080ce810 con 0x7f2410103340 2024-04-25T00:41:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.746+0000 7f240f7fe640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23f80a2b90 0x7f23f80a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f24101027e0 tx=0x7f2400009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.751+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2408021070 con 0x7f2410103340 2024-04-25T00:41:54.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.934+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2410062340 con 0x7f2410103340 2024-04-25T00:41:54.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.934+0000 7f240cff9640 1 -- 172.21.15.31:0/396821209 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f240809b740 con 0x7f2410103340 2024-04-25T00:41:54.935 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:41:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23f80a2b90 msgr2=0x7f23f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23f80a2b90 0x7f23f80a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f24101027e0 tx=0x7f2400009500 comp rx=0 tx=0).stop 2024-04-25T00:41:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 msgr2=0x7f241019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 0x7f241019ef50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f240800d950 tx=0x7f240800de10 comp rx=0 tx=0).stop 2024-04-25T00:41:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 shutdown_connections 2024-04-25T00:41:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f23f80a2b90 0x7f23f80a5050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2410103340 0x7f241019ef50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2410102980 0x7f2410197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 --2- 172.21.15.31:0/396821209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2410101790 0x7f24101979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.936+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 >> 172.21.15.31:0/396821209 conn(0x7f24100fcd20 msgr2=0x7f2410105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.937+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 shutdown_connections 2024-04-25T00:41:54.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:54.937+0000 7f2415b77640 1 -- 172.21.15.31:0/396821209 wait complete. 2024-04-25T00:41:54.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 -- 172.21.15.31:0/2490680259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048102280 msgr2=0x7fd048104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 --2- 172.21.15.31:0/2490680259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048102280 0x7fd048104670 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd030009900 tx=0x7fd03002a410 comp rx=0 tx=0).stop 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 -- 172.21.15.31:0/2490680259 shutdown_connections 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 --2- 172.21.15.31:0/2490680259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0481074e0 0x7fd0481098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 --2- 172.21.15.31:0/2490680259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd048104bb0 0x7fd048106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 --2- 172.21.15.31:0/2490680259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048102280 0x7fd048104670 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 -- 172.21.15.31:0/2490680259 >> 172.21.15.31:0/2490680259 conn(0x7fd0480fbc30 msgr2=0x7fd0480fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 -- 172.21.15.31:0/2490680259 shutdown_connections 2024-04-25T00:41:55.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.053+0000 7fd04e361640 1 -- 172.21.15.31:0/2490680259 wait complete. 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 Processor -- start 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 -- start start 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd048102280 0x7fd0481979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 0x7fd048197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0481074e0 0x7fd04819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd04810d310 con 0x7fd048104bb0 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd04810d190 con 0x7fd0481074e0 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd04e361640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd04810d490 con 0x7fd048102280 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd0477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 0x7fd048197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd0477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 0x7fd048197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36940/0 (socket says 172.21.15.31:36940) 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.054+0000 7fd0477fe640 1 -- 172.21.15.31:0/2731228737 learned_addr learned my addr 172.21.15.31:0/2731228737 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd047fff640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd048102280 0x7fd0481979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd04c8fe640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0481074e0 0x7fd04819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0477fe640 1 -- 172.21.15.31:0/2731228737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd048102280 msgr2=0x7fd0481979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0477fe640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd048102280 0x7fd0481979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0477fe640 1 -- 172.21.15.31:0/2731228737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0481074e0 msgr2=0x7fd04819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0477fe640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0481074e0 0x7fd04819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0477fe640 1 -- 172.21.15.31:0/2731228737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04819f5e0 con 0x7fd048104bb0 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd047fff640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd048102280 0x7fd0481979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0477fe640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 0x7fd048197f00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd03800d770 tx=0x7fd03800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038017070 con 0x7fd048104bb0 2024-04-25T00:41:55.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd038009c30 con 0x7fd048104bb0 2024-04-25T00:41:55.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04819f8d0 con 0x7fd048104bb0 2024-04-25T00:41:55.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03800f620 con 0x7fd048104bb0 2024-04-25T00:41:55.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.055+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04819fe20 con 0x7fd048104bb0 2024-04-25T00:41:55.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.056+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0480fd720 con 0x7fd048104bb0 2024-04-25T00:41:55.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.057+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd03801a070 con 0x7fd048104bb0 2024-04-25T00:41:55.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.058+0000 7fd0457fa640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd0340a2b90 0x7fd0340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.058+0000 7fd047fff640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd0340a2b90 0x7fd0340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.058+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd0380cec50 con 0x7fd048104bb0 2024-04-25T00:41:55.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.058+0000 7fd047fff640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd0340a2b90 0x7fd0340a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd03002a930 tx=0x7fd030005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:55.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.062+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd0380d4020 con 0x7fd048104bb0 2024-04-25T00:41:55.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.253+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd048062370 con 0x7fd048104bb0 2024-04-25T00:41:55.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.253+0000 7fd0457fa640 1 -- 172.21.15.31:0/2731228737 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fd038009de0 con 0x7fd048104bb0 2024-04-25T00:41:55.254 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:41:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.255+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd0340a2b90 msgr2=0x7fd0340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.255+0000 7fd04e361640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd0340a2b90 0x7fd0340a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd03002a930 tx=0x7fd030005fd0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.255+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 msgr2=0x7fd048197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.255+0000 7fd04e361640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 0x7fd048197f00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd03800d770 tx=0x7fd03800dc30 comp rx=0 tx=0).stop 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 shutdown_connections 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0481074e0 0x7fd04819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fd0340a2b90 0x7fd0340a5050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd048104bb0 0x7fd048197f00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 --2- 172.21.15.31:0/2731228737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd048102280 0x7fd0481979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 >> 172.21.15.31:0/2731228737 conn(0x7fd0480fbc30 msgr2=0x7fd048102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 shutdown_connections 2024-04-25T00:41:55.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.256+0000 7fd04e361640 1 -- 172.21.15.31:0/2731228737 wait complete. 2024-04-25T00:41:55.266 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:54 smithi031 ceph-mon[29167]: pgmap v701: 97 pgs: 97 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:41:55.266 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1862737912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:41:55.266 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3833496132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:41:55.266 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:54 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/396821209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:41:55.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 -- 172.21.15.31:0/1192414560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 msgr2=0x7fe4d8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/1192414560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d8101bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009ca0 tx=0x7fe4d002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 -- 172.21.15.31:0/1192414560 shutdown_connections 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/1192414560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 0x7fe4d8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/1192414560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4d81029a0 0x7fe4d8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/1192414560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d8101bd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 -- 172.21.15.31:0/1192414560 >> 172.21.15.31:0/1192414560 conn(0x7fe4d80fcd40 msgr2=0x7fe4d80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 -- 172.21.15.31:0/1192414560 shutdown_connections 2024-04-25T00:41:55.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.369+0000 7fe4dcc31640 1 -- 172.21.15.31:0/1192414560 wait complete. 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 Processor -- start 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 -- start start 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4d81029a0 0x7fe4d8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 0x7fe4d8074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810d380 con 0x7fe4d81017b0 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810d200 con 0x7fe4d8103360 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4dcc31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810d500 con 0x7fe4d81029a0 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d80754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36956/0 (socket says 172.21.15.31:36956) 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d659c640 1 -- 172.21.15.31:0/2688384610 learned_addr learned my addr 172.21.15.31:0/2688384610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d6d9d640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 0x7fe4d8074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d5d9b640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4d81029a0 0x7fe4d8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d659c640 1 -- 172.21.15.31:0/2688384610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4d81029a0 msgr2=0x7fe4d8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.370+0000 7fe4d659c640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4d81029a0 0x7fe4d8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4d659c640 1 -- 172.21.15.31:0/2688384610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 msgr2=0x7fe4d8074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4d659c640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 0x7fe4d8074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4d659c640 1 -- 172.21.15.31:0/2688384610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d8074900 con 0x7fe4d81017b0 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4d6d9d640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 0x7fe4d8074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4d659c640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d80754f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe4d002aeb0 tx=0x7fe4d0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004240 con 0x7fe4d81017b0 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d0003e10 con 0x7fe4d81017b0 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d81a1b50 con 0x7fe4d81017b0 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d003b410 con 0x7fe4d81017b0 2024-04-25T00:41:55.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.371+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81a20a0 con 0x7fe4d81017b0 2024-04-25T00:41:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.372+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d8108a70 con 0x7fe4d81017b0 2024-04-25T00:41:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.373+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe4d0030650 con 0x7fe4d81017b0 2024-04-25T00:41:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.374+0000 7fe4c77fe640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe4c00a2b90 0x7fe4c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.374+0000 7fe4d5d9b640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe4c00a2b90 0x7fe4c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.374+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe4d00f08a0 con 0x7fe4d81017b0 2024-04-25T00:41:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.374+0000 7fe4d5d9b640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe4c00a2b90 0x7fe4c00a5050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005e10 tx=0x7fe4cc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.379+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe4d004d3a0 con 0x7fe4d81017b0 2024-04-25T00:41:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:54 smithi169 ceph-mon[33112]: pgmap v701: 97 pgs: 97 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:41:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1862737912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:41:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3833496132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:41:55.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:54 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/396821209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:41:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:54 smithi040 ceph-mon[32945]: pgmap v701: 97 pgs: 97 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-04-25T00:41:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1862737912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:41:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3833496132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:41:55.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:54 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/396821209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:41:55.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.615+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe4d8062340 con 0x7fe4d81017b0 2024-04-25T00:41:55.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.623+0000 7fe4c77fe640 1 -- 172.21.15.31:0/2688384610 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740369 (secure 0 0 0) 0x7fe4d00bd770 con 0x7fe4d81017b0 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 14150, 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2153348461 2024-04-25T00:41:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2153348461 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/2153348461", 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-04-25T00:21:13.218846+0000", 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24112, 2024-04-25T00:41:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:41:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T00:41:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T00:41:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T00:41:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T00:41:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T00:41:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T00:41:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T00:41:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T00:41:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:41:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T00:41:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T00:41:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T00:41:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:41:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:41:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:41:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T00:41:55.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:41:55.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T00:41:55.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T00:41:55.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.730 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T00:41:55.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T00:41:55.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.739 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.740 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.744 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.745 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.755 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:41:55.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:41:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:41:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:41:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:41:55.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.783 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.804 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T00:41:55.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.809 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T00:41:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.825 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:41:55.841 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:41:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T00:41:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.854 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:41:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.859 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.860 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.863 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T00:41:55.865 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.866 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T00:41:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T00:41:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.875 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.881 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:41:55.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.883 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.884 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.904 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.913 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.916 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.918 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T00:41:55.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.923 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.928 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.929 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.932 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T00:41:55.934 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.935 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:55.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.946 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.949 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:55.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.957 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:55.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T00:41:55.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.985 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:55.996 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:55.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:55.998 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.002 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T00:41:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T00:41:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T00:41:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T00:41:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T00:41:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:41:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T00:41:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T00:41:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T00:41:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:41:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24118, 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:41:56.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T00:41:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.124 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T00:41:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.128 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:41:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T00:41:56.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T00:41:56.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T00:41:56.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T00:41:56.148 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:41:56.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T00:41:56.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T00:41:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:41:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:41:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T00:41:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:41:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T00:41:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:41:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T00:41:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T00:41:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:41:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:41:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T00:41:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:41:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T00:41:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:41:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T00:41:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:41:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T00:41:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:41:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:41:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:41:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T00:41:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:41:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:41:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:41:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T00:41:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T00:41:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T00:41:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T00:41:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T00:41:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T00:41:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:41:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T00:41:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:41:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.368 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.371 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.379 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.382 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.385 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.387 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.388 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T00:41:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.392 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.400 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.403 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:41:56.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T00:41:56.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.415 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.419 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.430 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.431 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.456 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.459 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.463 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.468 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T00:41:56.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T00:41:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T00:41:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.491 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.493 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.495 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.496 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:41:56.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T00:41:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.506 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.508 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.513 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.518 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.520 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.521 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.522 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.525 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.526 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T00:41:56.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T00:41:56.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T00:41:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T00:41:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "restful" 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:41:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:41:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T00:41:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:41:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:41:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T00:41:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T00:41:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T00:41:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T00:41:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T00:41:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T00:41:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T00:41:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:41:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T00:41:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:41:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:41:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:41:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T00:41:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T00:41:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T00:41:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T00:41:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T00:41:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.740 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.767 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.769 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.783 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.794 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.804 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.806 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T00:41:56.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.835 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.847 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:41:56.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T00:41:56.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.858 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:41:56.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.865 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.868 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T00:41:56.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T00:41:56.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.878 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.883 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T00:41:56.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.891 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.899 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T00:41:56.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T00:41:56.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T00:41:56.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.914 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.922 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.924 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.927 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:41:56.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T00:41:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T00:41:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T00:41:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T00:41:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T00:41:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T00:41:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 3, 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 128753718 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3409453448 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4147807999 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T00:41:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 209666436 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T00:41:56.986 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:41:56.987 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:55 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2731228737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:41:56.987 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:55 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2688384610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:41:56.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:55 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2731228737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:41:56.988 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:55 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2688384610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:41:56.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe4c00a2b90 msgr2=0x7fe4c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe4c00a2b90 0x7fe4c00a5050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005e10 tx=0x7fe4cc005da0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 msgr2=0x7fe4d80754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d80754f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe4d002aeb0 tx=0x7fe4d0003c20 comp rx=0 tx=0).stop 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 shutdown_connections 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d8103360 0x7fe4d8074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4d81029a0 0x7fe4d8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe4c00a2b90 0x7fe4c00a5050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 --2- 172.21.15.31:0/2688384610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d81017b0 0x7fe4d80754f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 >> 172.21.15.31:0/2688384610 conn(0x7fe4d80fcd40 msgr2=0x7fe4d8105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 shutdown_connections 2024-04-25T00:41:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:41:55.629+0000 7fe4dcc31640 1 -- 172.21.15.31:0/2688384610 wait complete. 2024-04-25T00:41:56.995 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T00:41:56.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T00:41:56.996 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:55 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2731228737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:41:56.996 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:55 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2688384610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:41:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:41:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:56 smithi031 ceph-mon[29167]: pgmap v702: 97 pgs: 97 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:41:57.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:56 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:41:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:56 smithi169 ceph-mon[33112]: pgmap v702: 97 pgs: 97 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:41:57.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:56 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:41:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:56 smithi040 ceph-mon[32945]: pgmap v702: 97 pgs: 97 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T00:41:57.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:56 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: pgmap v703: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:41:58.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:41:58 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: pgmap v703: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:41:58.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:41:58 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: pgmap v703: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:41:58 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 810, 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 811, 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 272155, 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 257263, 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.094511153, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.039251709, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.020308545, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.024265761, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 809, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.045719726, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000056513 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1503, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1670, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 383725, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 855949, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.112211737, 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.036771701, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.027462301, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.038841238, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1668, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.054696470, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000032791 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 378256 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 244, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 236, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 784388, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 139333, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.039352370, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.015781709, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.019806366, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004753941, 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 236, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.013514861, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000057266 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 783040, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 138368 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3909, 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.101765800, 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000026033 2024-04-25T00:41:59.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2024-04-25T00:41:59.155 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 0, 2024-04-25T00:41:59.156 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2024-04-25T00:41:59.157 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:41:59.158 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2024-04-25T00:41:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2024-04-25T00:41:59.160 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2024-04-25T00:41:59.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 3, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2024-04-25T00:41:59.162 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 2, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 2, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 4195385, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 4194892, 2024-04-25T00:41:59.163 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 3 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 3, 2024-04-25T00:41:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 3, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 1, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 1, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 52652, 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 215420 2024-04-25T00:41:59.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.170 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.171 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.172 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.173 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.173 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2024-04-25T00:41:59.173 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.173 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.174 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.175 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.177 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.178 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.179 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:41:59.180 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 1, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 1, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 0, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 1, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2024-04-25T00:41:59.181 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 1845, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:41:59.182 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:41:59.183 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 197520, 2024-04-25T00:41:59.184 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 56, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 7728, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 163, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T00:41:59.185 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 8464, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 13, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 2314, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2024-04-25T00:41:59.186 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 2314, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2314, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 1.575975517, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000681061 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2314, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 2315 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 2314, 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 2314, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 1499, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 813, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2024-04-25T00:41:59.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:41:59.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2024-04-25T00:41:59.190 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 91, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 4, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 6, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 6, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2024-04-25T00:41:59.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2024-04-25T00:41:59.192 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed_ops": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 2553, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1195306, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:41:59.193 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 2553, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2553, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1195306, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:41:59.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2314, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 1086, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2314, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1086, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.195 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2314, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2314, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2314, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2314, 2024-04-25T00:41:59.196 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:41:59.197 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55cc3f19e6e0": { 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:41:59.198 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:41:59.199 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:41:59.949 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T00:41:59.949 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T00:41:59.949 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-25T00:41:59.949 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-25T00:41:59.949 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T00:41:59.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T00:42:00.209 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:42:00.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:00 smithi169 ceph-mon[33112]: pgmap v704: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-25T00:42:00.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:00 smithi040 ceph-mon[32945]: pgmap v704: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-25T00:42:01.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:00 smithi031 ceph-mon[29167]: pgmap v704: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001449352, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000265801, 2024-04-25T00:42:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 236, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 229, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 760304, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 140032, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.038546333, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.015464853, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.020193384, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004704277, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 229, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.012591115, 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000054983 2024-04-25T00:42:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 760304, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 140032 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 173, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 108064, 2024-04-25T00:42:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014138325, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.011347575, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000759898, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 171, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.009599785, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000056139 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 108064 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148967, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T00:42:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 233, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 724487, 2024-04-25T00:42:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 233, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 233, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 724487, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:42:02.721 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:02 smithi031 ceph-mon[29167]: pgmap v705: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:42:02.752 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T00:42:02.752 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T00:42:02.752 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T00:42:02.752 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T00:42:02.753 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T00:42:02.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T00:42:02.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:02 smithi169 ceph-mon[33112]: pgmap v705: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:42:02.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:02 smithi040 ceph-mon[32945]: pgmap v705: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T00:42:03.006 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 229, 2024-04-25T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 224, 2024-04-25T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 756448, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 137664, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.040112429, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.015540732, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.020737757, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004808457, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 223, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.011911950, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000053416 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 756448, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 137664 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 170, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 107536, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.013754984, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.010333680, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001372139, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 168, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.009808708, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000058385 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 107536 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3040, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.000711164, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000040808, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000658850, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000695747, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000231915 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149007, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:04.776 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 230, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 724097, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 230, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 230, 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 724097, 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:42:04.779 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:04 smithi031 ceph-mon[29167]: pgmap v706: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T00:42:04.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:04 smithi169 ceph-mon[33112]: pgmap v706: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T00:42:04.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:04 smithi040 ceph-mon[32945]: pgmap v706: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T00:42:05.498 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T00:42:05.498 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T00:42:05.499 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T00:42:05.499 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T00:42:05.499 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T00:42:05.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T00:42:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:42:06.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:06 smithi169 ceph-mon[33112]: pgmap v707: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T00:42:06.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:06 smithi040 ceph-mon[32945]: pgmap v707: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T00:42:06.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:06 smithi031 ceph-mon[29167]: pgmap v707: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 306, 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 299, 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 813843, 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 203320, 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 12, 2024-04-25T00:42:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 6, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.044333335, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.013866204, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.025779873, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.005931598, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 295, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.018134884, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000061474 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 24, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000006718, 2024-04-25T00:42:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000279 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 808016, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 147840 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 13, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 187, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3451, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 388391, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.017894501, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.011065464, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.004687115, 2024-04-25T00:42:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000293071, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 185, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.045350186, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000245136 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 382720 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1171, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 775, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 294494, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 281317, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.087781465, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.036772228, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.053738248, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004875281, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 777, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.042065452, 2024-04-25T00:42:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000054138 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1823, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000422412, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000231 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2024-04-25T00:42:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 35, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.002253427, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000064383 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000007413, 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000003706 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 1, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 1, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 11, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 17, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 17, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 13, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 4, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 3, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 9, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 4, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 1, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 4, 2024-04-25T00:42:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 5, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 298, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2024-04-25T00:42:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 12, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 8, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 9, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2024-04-25T00:42:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2024-04-25T00:42:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 3, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 3, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 2, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 2, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 2, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2024-04-25T00:42:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 4195385, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 0, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.003561931, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001780965 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 0 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 21, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 23, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 2, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 20, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 23, 2024-04-25T00:42:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 3, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 17, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 21, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 4, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 54972, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 223612 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 1, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 1, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 9, 2024-04-25T00:42:07.526 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.529 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2024-04-25T00:42:07.530 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.534 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.535 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.536 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 1, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 1, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 0, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 1, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 699, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:07.538 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:07.539 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T00:42:07.540 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 222275, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 71, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 163, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 84765, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 333, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2024-04-25T00:42:07.542 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 37, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 11293, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 33, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 37, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 37, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.213996623, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.005783692 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 37, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 59 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 37, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 0, 2024-04-25T00:42:07.543 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 37, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 10, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 2, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 6, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 6, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 20, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2024-04-25T00:42:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2024-04-25T00:42:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 91, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 4, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 8, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 8, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 14, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 0, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 1 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 1, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2024-04-25T00:42:07.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.548 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2024-04-25T00:42:07.548 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.550 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 1486, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 960878, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 1486, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 1486, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 960878, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.551 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 37, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 11293, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 37, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 11293, 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:07.552 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 37, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 37, 2024-04-25T00:42:07.553 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 37, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 37, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:07.554 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x56338134e500": { 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T00:42:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 3, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1081, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 3, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1081, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T00:42:07.556 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T00:42:08.357 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-04-25T00:42:08.357 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-04-25T00:42:08.357 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=0 2024-04-25T00:42:08.357 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=12 2024-04-25T00:42:08.358 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T00:42:08.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T00:42:08.619 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:42:08.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:08 smithi040 ceph-mon[32945]: pgmap v708: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:42:08.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:08 smithi169 ceph-mon[33112]: pgmap v708: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:42:09.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:08 smithi031 ceph-mon[29167]: pgmap v708: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-25T00:42:10.613 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 309, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 299, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 824545, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 202969, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 13, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 7, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.047871600, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.015787883, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.026300260, 2024-04-25T00:42:10.614 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.006305993, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 294, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.022188702, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000075471 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 39, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000006525, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000167 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 818016, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 145120 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.615 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1185, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 959, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 297026, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 675728, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.098387006, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.042221395, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.057162576, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.005440056, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 961, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.056343959, 2024-04-25T00:42:10.616 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000058630 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1858, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000403419, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000217 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 383456 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 20, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 20, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 7449, 2024-04-25T00:42:10.617 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 13426, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.004421729, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.000309407, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.003124169, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000856673, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 19, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.019030345, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001001597 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.618 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:10.619 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 43, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.001788198, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000041586 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000009111, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000004555 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 4, 2024-04-25T00:42:10.620 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 3, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.002416475, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000805491 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 1, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 11, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 17, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 1, 2024-04-25T00:42:10.621 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 16, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 13, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 4, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 2, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 2, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 3, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 16, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 9, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 6, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:10.622 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 5, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 298, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:10.623 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:10.624 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 1 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 14, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 9, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 10, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-04-25T00:42:10.625 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T00:42:10.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 5, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 5, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 2, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-04-25T00:42:10.627 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 4195917, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 0, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.006700629, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001675157 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 0 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 21, 2024-04-25T00:42:10.628 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 23, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 2, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 20, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 23, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 3, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 17, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 21, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 4, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 2, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 3, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 1, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 51700, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 223612 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T00:42:10.629 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 8, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 4, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 14, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 1, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.002416475, 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000805491 2024-04-25T00:42:10.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.633 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T00:42:10.634 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.634 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.634 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.634 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.637 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:10.638 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T00:42:10.639 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 2, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 2, 2024-04-25T00:42:10.641 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 701, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:10.642 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:10.643 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 247329, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 88, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:10.644 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 163, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 85565, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 345, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 41, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 12070, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 28, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 41, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 41, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.240996776, 2024-04-25T00:42:10.646 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.005877970 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 41, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 64 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 41, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 0, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 41, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 0, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 10, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 0, 2024-04-25T00:42:10.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 4, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 7, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 6, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 20, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T00:42:10.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T00:42:10.649 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 91, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 4, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 10, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 10, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 16, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 0, 2024-04-25T00:42:10.650 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 1 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 1, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T00:42:10.651 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 1510, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 976044, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 1510, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:10.652 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 1510, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 976044, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:10.653 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 41, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 12070, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 41, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 12070, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T00:42:10.654 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 41, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 41, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 41, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 41, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.656 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55d2acbd8500": { 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 5, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 1613, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 5, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 4, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1613, 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:10.657 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:10.658 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:10.658 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:10.658 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.658 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:10.658 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T00:42:10.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:10 smithi031 ceph-mon[29167]: pgmap v709: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:42:10.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:10 smithi169 ceph-mon[33112]: pgmap v709: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:42:10.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:10 smithi040 ceph-mon[32945]: pgmap v709: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T00:42:11.408 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T00:42:11.409 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T00:42:11.409 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-04-25T00:42:11.409 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=14 2024-04-25T00:42:11.409 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T00:42:11.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T00:42:11.663 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:42:12.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:12 smithi169 ceph-mon[33112]: pgmap v710: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T00:42:12.894 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:12 smithi040 ceph-mon[32945]: pgmap v710: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T00:42:13.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:12 smithi031 ceph-mon[29167]: pgmap v710: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T00:42:13.586 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 250, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 239, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 808130, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 142724, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.044045026, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.015244315, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.023473254, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.006354258, 2024-04-25T00:42:13.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 238, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.013500408, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000056724 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 807296, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 142400 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 709, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 883, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 174350, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 609485, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.074464475, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.024820975, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.016705352, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.025533168, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 881, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.029840689, 2024-04-25T00:42:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000033871 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 384336 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1660, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1660, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 499578, 2024-04-25T00:42:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 527114, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.162461932, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.059011525, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.040886808, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.046971582, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1659, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.068170470, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000041091 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3930, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.107114217, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000027255 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-04-25T00:42:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T00:42:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T00:42:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-04-25T00:42:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-04-25T00:42:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T00:42:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 5, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-04-25T00:42:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 4195917, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 4195776, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 5 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T00:42:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 3, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 3, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 1, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 1, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 50028, 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-04-25T00:42:13.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.604 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T00:42:13.605 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.606 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T00:42:13.607 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.607 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.607 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.607 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T00:42:13.607 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.609 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T00:42:13.610 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.611 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.612 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.613 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 2, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 2985, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:13.616 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:13.617 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 189383, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 50, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T00:42:13.618 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 163, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 8464, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 13, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T00:42:13.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2365, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2365, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2365, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.084985761, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000458767 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T00:42:13.620 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2365, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2366 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2365, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2365, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1531, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 832, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-04-25T00:42:13.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:13.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T00:42:13.623 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 91, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 4, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 5, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 5, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T00:42:13.624 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T00:42:13.625 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2615, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 1230917, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:13.626 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2615, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2615, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1230917, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:13.627 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2365, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 1618, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2365, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1618, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2365, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2365, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2365, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2365, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:13.630 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x555e9faa66e0": { 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:13.631 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:13.632 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T00:42:14.373 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T00:42:14.373 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T00:42:14.373 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T00:42:14.373 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T00:42:14.373 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T00:42:14.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T00:42:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:42:14.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:14 smithi169 ceph-mon[33112]: pgmap v711: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T00:42:14.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:14 smithi040 ceph-mon[32945]: pgmap v711: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T00:42:14.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:14 smithi031 ceph-mon[29167]: pgmap v711: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1076, 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1065, 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 1085232, 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 405064, 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2024-04-25T00:42:16.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.128672289, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.048411079, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.040945229, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.030576596, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1064, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.047754109, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000044881 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 807024, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 141760 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 7, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 177, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 3756, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 385130, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.011254527, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.007652514, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.001725534, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000088905, 2024-04-25T00:42:16.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 176, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.009892542, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000056207 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 384432 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1587, 2024-04-25T00:42:16.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1589, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 405908, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 504673, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.105348706, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.025533785, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.029916066, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.043016422, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1587, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.042545124, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000026808 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:16.395 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4068, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.097265998, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000023910 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T00:42:16.396 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 0, 2024-04-25T00:42:16.397 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:16.398 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:16.399 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:16.400 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:16.401 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.402 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 5, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-04-25T00:42:16.403 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 4195917, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 4195424, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 5 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 3, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 3, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 0, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 1, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 1, 2024-04-25T00:42:16.404 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 50004, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-04-25T00:42:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.408 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T00:42:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.411 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.412 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.413 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.414 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.415 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 1842, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:16.417 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:16.418 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:16.419 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 182374, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 48, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:16.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 163, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 8464, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 13, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T00:42:16.421 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2417, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2417, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2417, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.385981794, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000573430 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T00:42:16.422 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2417, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2418 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2417, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2417, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1580, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 835, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T00:42:16.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T00:42:16.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T00:42:16.425 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 91, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 4, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 4, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 4, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.426 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:16.427 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2666, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 1239217, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2666, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2666, 2024-04-25T00:42:16.428 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1239217, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2417, 2024-04-25T00:42:16.429 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 1618, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2417, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1618, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:16.430 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2417, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2417, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2417, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2417, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:16.432 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x5596751a46e0": { 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:16.433 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.434 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:16.434 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T00:42:16.645 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:16 smithi040 ceph-mon[32945]: pgmap v712: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:42:16.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:16 smithi169 ceph-mon[33112]: pgmap v712: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:42:17.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:16 smithi031 ceph-mon[29167]: pgmap v712: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:42:17.170 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T00:42:17.170 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T00:42:17.171 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T00:42:17.171 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T00:42:17.171 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T00:42:17.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T00:42:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:42:18.815 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:18 smithi040 ceph-mon[32945]: pgmap v713: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:18.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:18 smithi169 ceph-mon[33112]: pgmap v713: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:19.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:18 smithi031 ceph-mon[29167]: pgmap v713: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1659, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1661, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 496998, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 527563, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.161873488, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.056642198, 2024-04-25T00:42:19.244 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.039961706, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.049356849, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1659, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.060272898, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000036330 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.245 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 966, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 958, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 962328, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 372528, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.111525566, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.033092923, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.040651446, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.033853010, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 958, 2024-04-25T00:42:19.246 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.035731942, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000037298 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 786032, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 145408 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 2, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 179, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 428, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 386798, 2024-04-25T00:42:19.247 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.010924100, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.007658774, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.001346109, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000020342, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 177, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.010019220, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000056605 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 386480 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3885, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.106483673, 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000027408 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-04-25T00:42:19.251 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-04-25T00:42:19.252 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:19.253 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:19.255 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T00:42:19.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 3, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-04-25T00:42:19.257 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 4195385, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 4194892, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 3 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 3, 2024-04-25T00:42:19.258 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 3, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 1, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 1, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 50088, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T00:42:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.260 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.261 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.263 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T00:42:19.264 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.265 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:19.267 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.268 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:19.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 1, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 0, 2024-04-25T00:42:19.272 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 1850, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:19.273 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:19.274 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T00:42:19.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 189339, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 50, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:19.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 163, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 8464, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 13, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.277 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2374, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2374, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2374, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.274983277, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000537061 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T00:42:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2374, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2375 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2374, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2374, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1539, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 833, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-04-25T00:42:19.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T00:42:19.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T00:42:19.281 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 91, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 4, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 5, 2024-04-25T00:42:19.282 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 5, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T00:42:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2623, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 1207920, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2623, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2623, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1207920, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2374, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 1086, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2374, 2024-04-25T00:42:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1086, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2374, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2374, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2374, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2374, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55f117b3c6e0": { 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T00:42:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T00:42:19.291 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T00:42:20.079 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T00:42:20.079 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T00:42:20.080 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T00:42:20.080 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-25T00:42:20.080 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T00:42:20.080 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T00:42:20.330 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:42:20.726 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:20 smithi169 ceph-mon[33112]: pgmap v714: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:20.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:20 smithi040 ceph-mon[32945]: pgmap v714: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:21.026 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:20 smithi031 ceph-mon[29167]: pgmap v714: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:22.204 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T00:42:22.204 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1192, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 800, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 303360, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 301683, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 8, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.109078774, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.051695567, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.059996691, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.004867055, 2024-04-25T00:42:22.205 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 803, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.054331017, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000067660 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1839, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000539543, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000293 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:42:22.206 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 305, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 293, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 821818, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 164442, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.045756590, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.016369281, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.024875685, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006447878, 2024-04-25T00:42:22.207 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 292, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.028751062, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000098462 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 17, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000002431, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000143 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 817152, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 150992 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.208 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 21, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 199, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 3961, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 439390, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 11, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 7, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.013682204, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007923281, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.003918845, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000154612, 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 193, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.013461335, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000069747 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 26, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000004879, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000187 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 387568 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T00:42:22.210 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 49, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001815184, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000037044 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:22.211 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000006709, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000003354 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 6, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 4, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001312067, 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000328016 2024-04-25T00:42:22.212 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 2, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 17, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 1, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 16, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:22.213 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 13, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 4, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 1, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 1, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 3, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 15, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 11, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 4, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:22.214 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 5, 2024-04-25T00:42:22.215 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 298, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:22.216 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 12, 2024-04-25T00:42:22.217 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 8, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 9, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:22.218 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T00:42:22.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 5, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 5, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T00:42:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195917, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.007687012, 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001921753 2024-04-25T00:42:22.221 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 21, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 23, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 2, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 20, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 23, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 3, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 17, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 21, 2024-04-25T00:42:22.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 4, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 1, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 4, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 3, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 54192, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223612 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 6, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 6, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 14, 2024-04-25T00:42:22.223 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000648179, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000324089 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:22.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000663888, 2024-04-25T00:42:22.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000331944 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:22.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T00:42:22.228 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.229 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.229 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T00:42:22.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.229 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:22.231 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.232 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.233 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.234 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T00:42:22.235 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.236 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 3, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 1, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 711, 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:22.237 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:22.238 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 231398, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 82, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 176, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 2, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 163, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:22.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 85197, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 343, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T00:42:22.242 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 43, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 12181, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 31, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 43, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 43, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.253994603, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005906851 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 43, 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 68 2024-04-25T00:42:22.243 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 43, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 43, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 4, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 7, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T00:42:22.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 6, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:22.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T00:42:22.246 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T00:42:22.247 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 91, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 4, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 8, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 8, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 19, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 2 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.248 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:22.249 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 1514, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 975291, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:22.250 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 1514, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1514, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 975291, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:22.251 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 43, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 12181, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 43, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 12181, 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:22.252 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 43, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 43, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 43, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 43, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T00:42:22.255 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x556f940e6500": { 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 5, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1613, 2024-04-25T00:42:22.256 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 5, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1613, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:22.257 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T00:42:22.873 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:22 smithi169 ceph-mon[33112]: pgmap v715: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:42:22.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:22 smithi040 ceph-mon[32945]: pgmap v715: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:42:22.901 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T00:42:22.901 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T00:42:22.901 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T00:42:22.901 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=12 2024-04-25T00:42:22.901 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T00:42:22.901 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T00:42:23.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:22 smithi031 ceph-mon[29167]: pgmap v715: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T00:42:23.140 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:42:24.878 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:24 smithi169 ceph-mon[33112]: pgmap v716: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T00:42:24.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:24 smithi040 ceph-mon[32945]: pgmap v716: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T00:42:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T00:42:24.913 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:24.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1737, 2024-04-25T00:42:24.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1739, 2024-04-25T00:42:24.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 4706049, 2024-04-25T00:42:24.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 552376, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.178653627, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.053478860, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.050994268, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.057758808, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1737, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.067304189, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000038747 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:24.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 298, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 473, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 73894, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 482256, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.040152668, 2024-04-25T00:42:24.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.018262764, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.007317894, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.010115410, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 471, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.025729814, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000054628 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:42:24.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 387744 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1346, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1333, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 4340244, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 491231, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 3, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.141845100, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.052783433, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.046866504, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.036116692, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1333, 2024-04-25T00:42:24.917 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.050766332, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000038084 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 811312, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 144928 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T00:42:24.918 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5335, 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.151155708, 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000028332 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:24.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T00:42:24.920 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 18, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 4, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 14, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 11, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-04-25T00:42:24.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:24.922 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T00:42:24.924 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:24.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 2389, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T00:42:24.926 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 54, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 11617865, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 11644823, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 11641401, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 2389 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 15, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 18, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 3, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 10, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 12, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 2, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 18, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 20, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 2, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-25T00:42:24.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 51528, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215420 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.929 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.930 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T00:42:24.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T00:42:24.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.934 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 3, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 2, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 2951, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T00:42:24.940 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:24.941 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:24.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 189463, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 52, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:24.943 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 163, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 58786, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 272, 2024-04-25T00:42:24.944 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 3123, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 26, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 4, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 3123, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3123, 2024-04-25T00:42:24.945 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.462968812, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000468449 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3123, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3124 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 3123, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 3123, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1653, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-04-25T00:42:24.946 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 1467, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 2, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:24.947 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T00:42:24.948 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 91, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 4, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 5, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 5, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-04-25T00:42:24.949 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:24.950 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 3377, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 8800264, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 3377, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3377, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 8800264, 2024-04-25T00:42:24.951 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 3123, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 7450529, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3123, 2024-04-25T00:42:24.952 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 7450529, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:24.953 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 3123, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 3123, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3123, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 3123, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:24.954 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x556f408c86e0": { 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:24.956 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T00:42:25.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:24 smithi031 ceph-mon[29167]: pgmap v716: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T00:42:25.821 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T00:42:25.821 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T00:42:25.821 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T00:42:25.821 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T00:42:25.821 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T00:42:25.821 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T00:42:26.062 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:42:26.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:26 smithi169 ceph-mon[33112]: pgmap v717: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:26.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:26 smithi040 ceph-mon[32945]: pgmap v717: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:27.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:26 smithi031 ceph-mon[29167]: pgmap v717: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:27.785 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4270, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 4658, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1165393, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 9113034, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 23, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.407202899, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.156986964, 2024-04-25T00:42:27.786 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.116540146, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.104516361, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4656, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.638625772, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000137161 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1860, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000519991, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000279 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.787 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 3247, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 6027, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1129746, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 2526480, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 15, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.443621273, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.174879407, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.184297870, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.082127260, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 6026, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.294970926, 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000048949 2024-04-25T00:42:27.788 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 7551, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001932220, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000255 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 53936, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 423728 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2311, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 2996, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1519371, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 2028116, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 102, 2024-04-25T00:42:27.789 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 7, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.288054120, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.087106319, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.135373097, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.059270015, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2995, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.181333392, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000060545 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3900, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001243335, 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000318 2024-04-25T00:42:27.790 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 859632, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 150112 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5819, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.920090923, 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000329969 2024-04-25T00:42:27.791 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 24, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000618987, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000025791 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 645, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 645, 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T00:42:27.792 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 645, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3.242839162, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005027657 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 353, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 25, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 4, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 21, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:27.793 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 18, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 2, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 2, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 664, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 654, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T00:42:27.794 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:27.795 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 613, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 1211, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 599, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 620, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 603, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 2, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 25 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T00:42:27.796 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 16, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 35, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 3, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 3, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:27.797 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 2389, 2024-04-25T00:42:27.798 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 2375, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 2375, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 14, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 50, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 55, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 54, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 54, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 1, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 1, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 1, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 11617865, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 11644823, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T00:42:27.799 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2373, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 5.209549012, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002195343 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 23, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 26, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 3, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 18, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 20, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 2, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 25, 2024-04-25T00:42:27.800 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 28, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 3, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 2, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 17, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 15, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 56456, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 231804 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 1868, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 1, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 645, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 20, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 2, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5, 2024-04-25T00:42:27.801 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003865465, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000773093 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 17, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.004507586, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000265152 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.802 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.803 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.805 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 8, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.013084431, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001635553 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.806 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 599, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3.183353396, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005314446 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002246136, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001123068 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002018336, 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001009168 2024-04-25T00:42:27.807 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.012199169, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.006099584 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:27.808 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001096998, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001096998 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.809 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.810 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 8, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.018348178, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002293522 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002119467, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002119467 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:27.811 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 3, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 95, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 95, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 717, 2024-04-25T00:42:27.812 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:27.813 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 322738, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 128, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:27.815 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 163, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 97994, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 709, 2024-04-25T00:42:27.816 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 4161, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 7814586, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 37, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 4161, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4161, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 27.503414915, 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.006609808 2024-04-25T00:42:27.817 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4161, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 4915 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 4161, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 4159, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 333, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 28, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 2, 2024-04-25T00:42:27.818 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 2375, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 170, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 517, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 56, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T00:42:27.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 599, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T00:42:27.820 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 91, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 5, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 12, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 821, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 14 2024-04-25T00:42:27.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 2, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 2, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 3, 2024-04-25T00:42:27.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 3, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 9824, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 2903348, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 9824, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 9824, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2903348, 2024-04-25T00:42:27.823 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 4161, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 7814889, 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4161, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 7814889, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:27.825 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 4161, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 4161, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4161, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 4161, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:27.826 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 3, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 303, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 3, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 303, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x5565774ba500": { 2024-04-25T00:42:27.827 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2389, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 7450519, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2389, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2373, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 7450519, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.828 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:27.829 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T00:42:28.587 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T00:42:28.587 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T00:42:28.587 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=16 2024-04-25T00:42:28.587 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-25T00:42:28.587 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T00:42:28.587 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 6c091b24-0299-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T00:42:28.827 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:42:28.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:28 smithi169 ceph-mon[33112]: pgmap v718: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:28.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:28 smithi040 ceph-mon[32945]: pgmap v718: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:29.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:28 smithi031 ceph-mon[29167]: pgmap v718: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:30.567 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T00:42:30.567 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:42:30.567 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 27, 2024-04-25T00:42:30.567 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 33, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 5588, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 55558, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 13, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.005456902, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000620502, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.003414166, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000246426, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 27, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008004035, 2024-04-25T00:42:30.568 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000296445 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 24, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000004828, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000201 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1438, 2024-04-25T00:42:30.569 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1045, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1081547, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 430132, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.153502728, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.066438138, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.093103521, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.010756449, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1048, 2024-04-25T00:42:30.570 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.061289080, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000058481 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1791, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000401321, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000224 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 800016, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 152320 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:42:30.571 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 20, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 197, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 7281, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 397877, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.012068644, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007640823, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.002241545, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000432031, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 196, 2024-04-25T00:42:30.572 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.025624837, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000130738 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 390512 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.573 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 39, 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001656966, 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000042486 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000009710, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000004855 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-25T00:42:30.575 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T00:42:30.576 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 16, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 16, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 13, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 4, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 3, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 6, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 3, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 3, 2024-04-25T00:42:30.577 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T00:42:30.578 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 5, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 298, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:42:30.579 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 9, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 6, 2024-04-25T00:42:30.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 7, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:42:30.581 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.582 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 3, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 3, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-04-25T00:42:30.583 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195385, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003027676, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001513838 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.584 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 20, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 22, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 2, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 19, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 22, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 3, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 16, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 20, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 4, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T00:42:30.585 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 52088, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223612 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 10, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T00:42:30.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.587 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.589 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.591 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T00:42:30.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.596 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T00:42:30.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 1, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 0, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 719, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T00:42:30.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:42:30.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:42:30.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:42:30.602 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 230135, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 74, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7728, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 163, 2024-04-25T00:42:30.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 80557, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 316, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T00:42:30.604 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 37, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 11293, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 33, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 37, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 37, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.191995901, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005189078 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 37, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 59 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 37, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 37, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 2, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 6, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T00:42:30.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 6, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T00:42:30.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T00:42:30.608 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 91, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 4, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 9, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 9, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T00:42:30.609 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 14, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 1 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T00:42:30.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:42:30.611 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 1481, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 943680, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 1481, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1481, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 943680, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.612 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 37, 2024-04-25T00:42:30.613 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 11293, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 37, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 11293, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T00:42:30.614 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 37, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 37, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 37, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 37, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.615 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:30.616 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x560aa6f84500": { 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T00:42:30.617 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 3, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1081, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 3, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1081, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T00:42:30.618 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.619 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T00:42:30.619 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T00:42:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:30 smithi169 ceph-mon[33112]: pgmap v719: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:42:30.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:30 smithi040 ceph-mon[32945]: pgmap v719: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:42:31.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:30 smithi031 ceph-mon[29167]: pgmap v719: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:42:31.479 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T00:42:31.479 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T00:42:31.479 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T00:42:31.479 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=9 2024-04-25T00:42:31.479 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T00:42:31.969 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T00:42:31.969 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:42:31.969 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:42:31.992 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:42:31.992 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:42:32.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 -- 172.21.15.31:0/4243164018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00101890 msgr2=0x7efe00101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 --2- 172.21.15.31:0/4243164018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00101890 0x7efe00101d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009860 tx=0x7efdf002a340 comp rx=0 tx=0).stop 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 -- 172.21.15.31:0/4243164018 shutdown_connections 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 --2- 172.21.15.31:0/4243164018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00102250 0x7efe00108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 --2- 172.21.15.31:0/4243164018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00101890 0x7efe00101d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 --2- 172.21.15.31:0/4243164018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe001006a0 0x7efe00100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 -- 172.21.15.31:0/4243164018 >> 172.21.15.31:0/4243164018 conn(0x7efe000fbc30 msgr2=0x7efe000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.205+0000 7efe0822c640 1 -- 172.21.15.31:0/4243164018 shutdown_connections 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 -- 172.21.15.31:0/4243164018 wait complete. 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 Processor -- start 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 -- start start 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe001006a0 0x7efe0010afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 0x7efe00109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00102250 0x7efe00109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe0010d2f0 con 0x7efe00101890 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe0010d170 con 0x7efe001006a0 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe0822c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efe0010d470 con 0x7efe00102250 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe057c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 0x7efe00109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe057c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 0x7efe00109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53320/0 (socket says 172.21.15.31:53320) 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.206+0000 7efe057c7640 1 -- 172.21.15.31:0/4085414105 learned_addr learned my addr 172.21.15.31:0/4085414105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe067c9640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00102250 0x7efe00109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe05fc8640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe001006a0 0x7efe0010afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe057c7640 1 -- 172.21.15.31:0/4085414105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00102250 msgr2=0x7efe00109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe057c7640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00102250 0x7efe00109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe057c7640 1 -- 172.21.15.31:0/4085414105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe001006a0 msgr2=0x7efe0010afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe057c7640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe001006a0 0x7efe0010afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe057c7640 1 -- 172.21.15.31:0/4085414105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0010a460 con 0x7efe00101890 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe057c7640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 0x7efe00109660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efdf00057c0 tx=0x7efdf0030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0040030 con 0x7efe00101890 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe001a1b20 con 0x7efe00101890 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf0038040 con 0x7efe00101890 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0037070 con 0x7efe00101890 2024-04-25T00:42:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.207+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a1fe0 con 0x7efe00101890 2024-04-25T00:42:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.208+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00107ae0 con 0x7efe00101890 2024-04-25T00:42:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.209+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efdf0043070 con 0x7efe00101890 2024-04-25T00:42:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.210+0000 7efdf6ffd640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efdec0a2b90 0x7efdec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.210+0000 7efe05fc8640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efdec0a2b90 0x7efdec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.210+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efdf00eedb0 con 0x7efe00101890 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.211+0000 7efe05fc8640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efdec0a2b90 0x7efdec0a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005fd0 tx=0x7efdfc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.214+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efdf004a020 con 0x7efe00101890 2024-04-25T00:42:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.400+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe0002d330 con 0x7efe00101890 2024-04-25T00:42:32.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.401+0000 7efdf6ffd640 1 -- 172.21.15.31:0/4085414105 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7efdf00bbc80 con 0x7efe00101890 2024-04-25T00:42:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:42:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:32.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.403+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efdec0a2b90 msgr2=0x7efdec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.403+0000 7efe0822c640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efdec0a2b90 0x7efdec0a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005fd0 tx=0x7efdfc00b390 comp rx=0 tx=0).stop 2024-04-25T00:42:32.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.403+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 msgr2=0x7efe00109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.403+0000 7efe0822c640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 0x7efe00109660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efdf00057c0 tx=0x7efdf0030e00 comp rx=0 tx=0).stop 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.403+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 shutdown_connections 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe00102250 0x7efe00109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7efdec0a2b90 0x7efdec0a5050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe00101890 0x7efe00109660 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 --2- 172.21.15.31:0/4085414105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe001006a0 0x7efe0010afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 >> 172.21.15.31:0/4085414105 conn(0x7efe000fbc30 msgr2=0x7efe00104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 shutdown_connections 2024-04-25T00:42:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.404+0000 7efe0822c640 1 -- 172.21.15.31:0/4085414105 wait complete. 2024-04-25T00:42:32.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:42:32.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 -- 172.21.15.31:0/3557671801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 msgr2=0x7f722c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 --2- 172.21.15.31:0/3557671801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c108ae0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7224009ca0 tx=0x7f722402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 -- 172.21.15.31:0/3557671801 shutdown_connections 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 --2- 172.21.15.31:0/3557671801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c108ae0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 --2- 172.21.15.31:0/3557671801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 0x7f722c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 --2- 172.21.15.31:0/3557671801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 0x7f722c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 -- 172.21.15.31:0/3557671801 >> 172.21.15.31:0/3557671801 conn(0x7f722c0fbc30 msgr2=0x7f722c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 -- 172.21.15.31:0/3557671801 shutdown_connections 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 -- 172.21.15.31:0/3557671801 wait complete. 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 Processor -- start 2024-04-25T00:42:32.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.521+0000 7f72321c1640 1 -- start start 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f72321c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 0x7f722c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f72321c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 0x7f722c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f72321c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f72321c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f722c10d310 con 0x7f722c102250 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f72321c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f722c10d190 con 0x7f722c101890 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f72321c1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f722c10d490 con 0x7f722c1006a0 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53342/0 (socket says 172.21.15.31:53342) 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 -- 172.21.15.31:0/2415294836 learned_addr learned my addr 172.21.15.31:0/2415294836 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722b7fe640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 0x7f722c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f7222ffd640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 0x7f722c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 -- 172.21.15.31:0/2415294836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 msgr2=0x7f722c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 0x7f722c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 -- 172.21.15.31:0/2415294836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 msgr2=0x7f722c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 0x7f722c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 -- 172.21.15.31:0/2415294836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c19f590 con 0x7f722c102250 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f7222ffd640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 0x7f722c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722b7fe640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 0x7f722c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.522+0000 7f722bfff640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c19ef20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7224031040 tx=0x7f7224003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.523+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224032ec0 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.523+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72240044f0 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.523+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224030cb0 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.523+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c19f820 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.523+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c19fdc0 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.524+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722c1079e0 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.525+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7224004040 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.526+0000 7f72297fa640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72180a2db0 0x7f72180a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.526+0000 7f722b7fe640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72180a2db0 0x7f72180a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.526+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f72240f05d0 con 0x7f722c102250 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.526+0000 7f722b7fe640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72180a2db0 0x7f72180a5270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f722c1016f0 tx=0x7f721c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.530+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f72240bd4a0 con 0x7f722c102250 2024-04-25T00:42:32.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.716+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f722c062370 con 0x7f722c102250 2024-04-25T00:42:32.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.717+0000 7f72297fa640 1 -- 172.21.15.31:0/2415294836 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f72240bcda0 con 0x7f722c102250 2024-04-25T00:42:32.718 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72180a2db0 msgr2=0x7f72180a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72180a2db0 0x7f72180a5270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f722c1016f0 tx=0x7f721c005f90 comp rx=0 tx=0).stop 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 msgr2=0x7f722c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c19ef20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7224031040 tx=0x7f7224003ba0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 shutdown_connections 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f72180a2db0 0x7f72180a5270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f722c102250 0x7f722c19ef20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f722c101890 0x7f722c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 --2- 172.21.15.31:0/2415294836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c1006a0 0x7f722c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 >> 172.21.15.31:0/2415294836 conn(0x7f722c0fbc30 msgr2=0x7f722c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 shutdown_connections 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.719+0000 7f72321c1640 1 -- 172.21.15.31:0/2415294836 wait complete. 2024-04-25T00:42:32.730 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:32 smithi031 ceph-mon[29167]: pgmap v720: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:42:32.730 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:32 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4085414105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:32.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:42:32.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.837+0000 7f1514770640 1 -- 172.21.15.31:0/3256793853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c101870 msgr2=0x7f150c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.837+0000 7f1514770640 1 --2- 172.21.15.31:0/3256793853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c101870 0x7f150c101cf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009900 tx=0x7f14fc02a410 comp rx=0 tx=0).stop 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- 172.21.15.31:0/3256793853 shutdown_connections 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 --2- 172.21.15.31:0/3256793853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c102230 0x7f150c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 --2- 172.21.15.31:0/3256793853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c101870 0x7f150c101cf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 --2- 172.21.15.31:0/3256793853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c100680 0x7f150c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- 172.21.15.31:0/3256793853 >> 172.21.15.31:0/3256793853 conn(0x7f150c0fbbf0 msgr2=0x7f150c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- 172.21.15.31:0/3256793853 shutdown_connections 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- 172.21.15.31:0/3256793853 wait complete. 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 Processor -- start 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- start start 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c100680 0x7f150c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c101870 0x7f150c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 0x7f150c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10c190 con 0x7f150c102230 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10c010 con 0x7f150c101870 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.838+0000 7f1514770640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10c310 con 0x7f150c100680 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 0x7f150c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 0x7f150c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53490/0 (socket says 172.21.15.31:53490) 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 -- 172.21.15.31:0/3428344677 learned_addr learned my addr 172.21.15.31:0/3428344677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f151250c640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c100680 0x7f150c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1511d0b640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c101870 0x7f150c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 -- 172.21.15.31:0/3428344677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c100680 msgr2=0x7f150c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c100680 0x7f150c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 -- 172.21.15.31:0/3428344677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c101870 msgr2=0x7f150c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c101870 0x7f150c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 -- 172.21.15.31:0/3428344677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c19f660 con 0x7f150c102230 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1511d0b640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c101870 0x7f150c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1512d0d640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 0x7f150c19eff0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f150800cad0 tx=0x7f150800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150800ae20 con 0x7f150c102230 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c19f950 con 0x7f150c102230 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1508005020 con 0x7f150c102230 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150800f6b0 con 0x7f150c102230 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.839+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c19fea0 con 0x7f150c102230 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.840+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c1a0130 con 0x7f150c102230 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.841+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f15080051d0 con 0x7f150c102230 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.842+0000 7f15037fe640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14f80a2b90 0x7f14f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.842+0000 7f151250c640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14f80a2b90 0x7f14f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.842+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f15080cf510 con 0x7f150c102230 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.843+0000 7f151250c640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14f80a2b90 0x7f14f80a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1504005fd0 tx=0x7f1504005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:32.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:32.846+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1508014030 con 0x7f150c102230 2024-04-25T00:42:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:32 smithi169 ceph-mon[33112]: pgmap v720: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:42:32.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:32 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4085414105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:32 smithi040 ceph-mon[32945]: pgmap v720: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:42:32.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:32 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4085414105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.029+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f150c1a0300 con 0x7f150c102230 2024-04-25T00:42:33.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.029+0000 7f15037fe640 1 -- 172.21.15.31:0/3428344677 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f150809c440 con 0x7f150c102230 2024-04-25T00:42:33.030 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:42:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14f80a2b90 msgr2=0x7f14f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14f80a2b90 0x7f14f80a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1504005fd0 tx=0x7f1504005850 comp rx=0 tx=0).stop 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 msgr2=0x7f150c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 0x7f150c19eff0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f150800cad0 tx=0x7f150800cf90 comp rx=0 tx=0).stop 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 shutdown_connections 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f14f80a2b90 0x7f14f80a5050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150c102230 0x7f150c19eff0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f150c101870 0x7f150c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 --2- 172.21.15.31:0/3428344677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c100680 0x7f150c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 >> 172.21.15.31:0/3428344677 conn(0x7f150c0fbbf0 msgr2=0x7f150c068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 shutdown_connections 2024-04-25T00:42:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.031+0000 7f1514770640 1 -- 172.21.15.31:0/3428344677 wait complete. 2024-04-25T00:42:33.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:42:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/770606473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 msgr2=0x7f6b24100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/770606473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24100ac0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009900 tx=0x7f6b2002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/770606473 shutdown_connections 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/770606473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24102250 0x7f6b24108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/770606473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24101890 0x7f6b24101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/770606473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24100ac0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/770606473 >> 172.21.15.31:0/770606473 conn(0x7f6b240fbc30 msgr2=0x7f6b240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/770606473 shutdown_connections 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.148+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/770606473 wait complete. 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 Processor -- start 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 -- start start 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24101890 0x7f6b24102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24102250 0x7f6b241031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410c1d0 con 0x7f6b241006a0 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410c050 con 0x7f6b24101890 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2bc8c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410c350 con 0x7f6b24102250 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53516/0 (socket says 172.21.15.31:53516) 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 -- 172.21.15.31:0/3208069956 learned_addr learned my addr 172.21.15.31:0/3208069956 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b2a229640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24102250 0x7f6b241031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29227640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24101890 0x7f6b24102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 -- 172.21.15.31:0/3208069956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24102250 msgr2=0x7f6b241031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24102250 0x7f6b241031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 -- 172.21.15.31:0/3208069956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24101890 msgr2=0x7f6b24102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24101890 0x7f6b24102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 -- 172.21.15.31:0/3208069956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b24103a40 con 0x7f6b241006a0 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29227640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24101890 0x7f6b24102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.149+0000 7f6b29a28640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24104630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6b200314e0 tx=0x7f6b20031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.150+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2003f030 con 0x7f6b241006a0 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.150+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b200044b0 con 0x7f6b241006a0 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.150+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b241a1b50 con 0x7f6b241006a0 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.150+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20035070 con 0x7f6b241006a0 2024-04-25T00:42:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.150+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b241a20f0 con 0x7f6b241006a0 2024-04-25T00:42:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.151+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b241a22c0 con 0x7f6b241006a0 2024-04-25T00:42:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.152+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6b20042030 con 0x7f6b241006a0 2024-04-25T00:42:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.152+0000 7f6b1affd640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6b100a2c10 0x7f6b100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.153+0000 7f6b29227640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6b100a2c10 0x7f6b100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.153+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f6b200ef590 con 0x7f6b241006a0 2024-04-25T00:42:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.153+0000 7f6b29227640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6b100a2c10 0x7f6b100a50d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b241043a0 tx=0x7f6b14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:33.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.157+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6b200bc4c0 con 0x7f6b241006a0 2024-04-25T00:42:33.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.341+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b241a2490 con 0x7f6b241006a0 2024-04-25T00:42:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.341+0000 7f6b1affd640 1 -- 172.21.15.31:0/3208069956 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f6b200bbdc0 con 0x7f6b241006a0 2024-04-25T00:42:33.342 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6b100a2c10 msgr2=0x7f6b100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6b100a2c10 0x7f6b100a50d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b241043a0 tx=0x7f6b14005f90 comp rx=0 tx=0).stop 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 msgr2=0x7f6b24104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24104630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6b200314e0 tx=0x7f6b20031a30 comp rx=0 tx=0).stop 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 shutdown_connections 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24102250 0x7f6b241031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b24101890 0x7f6b24102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f6b100a2c10 0x7f6b100a50d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 --2- 172.21.15.31:0/3208069956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b241006a0 0x7f6b24104630 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 >> 172.21.15.31:0/3208069956 conn(0x7f6b240fbc30 msgr2=0x7f6b24069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 shutdown_connections 2024-04-25T00:42:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.343+0000 7f6b2bc8c640 1 -- 172.21.15.31:0/3208069956 wait complete. 2024-04-25T00:42:33.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 -- 172.21.15.31:0/1165892833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c102980 msgr2=0x7f3e4c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 --2- 172.21.15.31:0/1165892833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c102980 0x7f3e4c102e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009900 tx=0x7f3e4002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 -- 172.21.15.31:0/1165892833 shutdown_connections 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 --2- 172.21.15.31:0/1165892833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c103340 0x7f3e4c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 --2- 172.21.15.31:0/1165892833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c102980 0x7f3e4c102e00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 --2- 172.21.15.31:0/1165892833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c101790 0x7f3e4c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 -- 172.21.15.31:0/1165892833 >> 172.21.15.31:0/1165892833 conn(0x7f3e4c0fcd20 msgr2=0x7f3e4c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 -- 172.21.15.31:0/1165892833 shutdown_connections 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.458+0000 7f3e529f9640 1 -- 172.21.15.31:0/1165892833 wait complete. 2024-04-25T00:42:33.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 Processor -- start 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 -- start start 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 0x7f3e4c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c102980 0x7f3e4c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c103340 0x7f3e4c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c10d2c0 con 0x7f3e4c101790 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c10d140 con 0x7f3e4c102980 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e529f9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c10d440 con 0x7f3e4c103340 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 0x7f3e4c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 0x7f3e4c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53544/0 (socket says 172.21.15.31:53544) 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e4bfff640 1 -- 172.21.15.31:0/2604112604 learned_addr learned my addr 172.21.15.31:0/2604112604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e50f96640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c103340 0x7f3e4c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.459+0000 7f3e4b7fe640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c102980 0x7f3e4c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e4bfff640 1 -- 172.21.15.31:0/2604112604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c103340 msgr2=0x7f3e4c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e4bfff640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c103340 0x7f3e4c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e4bfff640 1 -- 172.21.15.31:0/2604112604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c102980 msgr2=0x7f3e4c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e4bfff640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c102980 0x7f3e4c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e4bfff640 1 -- 172.21.15.31:0/2604112604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c19f640 con 0x7f3e4c101790 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e4bfff640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 0x7f3e4c197a20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00e9c0 tx=0x7f3e3c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00cd80 con 0x7f3e4c101790 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3c0035f0 con 0x7f3e4c101790 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c19f930 con 0x7f3e4c101790 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c010690 con 0x7f3e4c101790 2024-04-25T00:42:33.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.460+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c19fe00 con 0x7f3e4c101790 2024-04-25T00:42:33.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.461+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c108bc0 con 0x7f3e4c101790 2024-04-25T00:42:33.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.462+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3e3c0037a0 con 0x7f3e4c101790 2024-04-25T00:42:33.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.463+0000 7f3e497fa640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e340a2b90 0x7f3e340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.463+0000 7f3e4b7fe640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e340a2b90 0x7f3e340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.463+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f3e3c0cf600 con 0x7f3e4c101790 2024-04-25T00:42:33.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.463+0000 7f3e4b7fe640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e340a2b90 0x7f3e340a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006010 tx=0x7f3e40000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:33.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.468+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e3c09c4d0 con 0x7f3e4c101790 2024-04-25T00:42:33.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.668+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3e4c062340 con 0x7f3e4c101790 2024-04-25T00:42:33.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.669+0000 7f3e497fa640 1 -- 172.21.15.31:0/2604112604 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+592 (secure 0 0 0) 0x7f3e3c09bdd0 con 0x7f3e4c101790 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3395718020 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3443312222 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3443312222 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3607070745 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3134587413 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3135101963 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1911777180 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3278889828 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1911777180 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4126189511 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:33.670 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2024-04-25T00:42:33.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e340a2b90 msgr2=0x7f3e340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e340a2b90 0x7f3e340a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006010 tx=0x7f3e40000bc0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 msgr2=0x7f3e4c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 0x7f3e4c197a20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00e9c0 tx=0x7f3e3c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 shutdown_connections 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c103340 0x7f3e4c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c102980 0x7f3e4c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f3e340a2b90 0x7f3e340a5050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 --2- 172.21.15.31:0/2604112604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e4c101790 0x7f3e4c197a20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 >> 172.21.15.31:0/2604112604 conn(0x7f3e4c0fcd20 msgr2=0x7f3e4c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 shutdown_connections 2024-04-25T00:42:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.671+0000 7f3e529f9640 1 -- 172.21.15.31:0/2604112604 wait complete. 2024-04-25T00:42:33.683 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:42:33.683 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:42:33.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 -- 172.21.15.31:0/353228059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c101890 msgr2=0x7ff35c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 --2- 172.21.15.31:0/353228059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c101890 0x7ff35c101d10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff350009900 tx=0x7ff35002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 -- 172.21.15.31:0/353228059 shutdown_connections 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 --2- 172.21.15.31:0/353228059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c102250 0x7ff35c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 --2- 172.21.15.31:0/353228059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c101890 0x7ff35c101d10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 --2- 172.21.15.31:0/353228059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c1006a0 0x7ff35c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 -- 172.21.15.31:0/353228059 >> 172.21.15.31:0/353228059 conn(0x7ff35c0fbc30 msgr2=0x7ff35c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 -- 172.21.15.31:0/353228059 shutdown_connections 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.840+0000 7ff361c88640 1 -- 172.21.15.31:0/353228059 wait complete. 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 Processor -- start 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 -- start start 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c1006a0 0x7ff35c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c101890 0x7ff35c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 0x7ff35c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c10c1b0 con 0x7ff35c102250 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c10c030 con 0x7ff35c101890 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff361c88640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c10c330 con 0x7ff35c1006a0 2024-04-25T00:42:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.841+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 0x7ff35c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 0x7ff35c19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53550/0 (socket says 172.21.15.31:53550) 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 -- 172.21.15.31:0/1659936113 learned_addr learned my addr 172.21.15.31:0/1659936113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35affd640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c101890 0x7ff35c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35b7fe640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c1006a0 0x7ff35c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 -- 172.21.15.31:0/1659936113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c1006a0 msgr2=0x7ff35c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c1006a0 0x7ff35c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 -- 172.21.15.31:0/1659936113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c101890 msgr2=0x7ff35c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c101890 0x7ff35c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 -- 172.21.15.31:0/1659936113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c19e4d0 con 0x7ff35c102250 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35affd640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c101890 0x7ff35c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35b7fe640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c1006a0 0x7ff35c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 0x7ff35c19de60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff35400eaa0 tx=0x7ff35400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35400cdb0 con 0x7ff35c102250 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c19e7c0 con 0x7ff35c102250 2024-04-25T00:42:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff354005020 con 0x7ff35c102250 2024-04-25T00:42:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540116d0 con 0x7ff35c102250 2024-04-25T00:42:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.842+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c19ece0 con 0x7ff35c102250 2024-04-25T00:42:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.843+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c107a90 con 0x7ff35c102250 2024-04-25T00:42:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.844+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff3540051d0 con 0x7ff35c102250 2024-04-25T00:42:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.845+0000 7ff358ff9640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3440a2b90 0x7ff3440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.845+0000 7ff35b7fe640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3440a2b90 0x7ff3440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.845+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff3540cf2d0 con 0x7ff35c102250 2024-04-25T00:42:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.845+0000 7ff35b7fe640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3440a2b90 0x7ff3440a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff35c1016f0 tx=0x7ff34c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:33.850+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff354016030 con 0x7ff35c102250 2024-04-25T00:42:33.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:33 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2415294836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:42:33.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:33 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3428344677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:42:33.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:33 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3208069956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:42:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:33 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2415294836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:42:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:33 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3428344677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:42:33.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:33 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3208069956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:33 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2415294836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:33 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3428344677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:33 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3208069956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.036+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff35c02d170 con 0x7ff35c102250 2024-04-25T00:42:34.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.037+0000 7ff358ff9640 1 -- 172.21.15.31:0/1659936113 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7ff35409c1a0 con 0x7ff35c102250 2024-04-25T00:42:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:42:34.039 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3440a2b90 msgr2=0x7ff3440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3440a2b90 0x7ff3440a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff35c1016f0 tx=0x7ff34c005f90 comp rx=0 tx=0).stop 2024-04-25T00:42:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 msgr2=0x7ff35c19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 0x7ff35c19de60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff35400eaa0 tx=0x7ff35400ef60 comp rx=0 tx=0).stop 2024-04-25T00:42:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 shutdown_connections 2024-04-25T00:42:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7ff3440a2b90 0x7ff3440a5050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c102250 0x7ff35c19de60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff35c101890 0x7ff35c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 --2- 172.21.15.31:0/1659936113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff35c1006a0 0x7ff35c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.039+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 >> 172.21.15.31:0/1659936113 conn(0x7ff35c0fbc30 msgr2=0x7ff35c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.040+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 shutdown_connections 2024-04-25T00:42:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.040+0000 7ff361c88640 1 -- 172.21.15.31:0/1659936113 wait complete. 2024-04-25T00:42:34.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:23:41.749279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:28:21.908076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:28:23.150234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:42:34.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:42:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 -- 172.21.15.31:0/981109792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 msgr2=0x7f5dec0ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 --2- 172.21.15.31:0/981109792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec0ffc90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009900 tx=0x7f5de002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 -- 172.21.15.31:0/981109792 shutdown_connections 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 --2- 172.21.15.31:0/981109792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec1001d0 0x7f5dec109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 --2- 172.21.15.31:0/981109792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec0ffc90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 --2- 172.21.15.31:0/981109792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dec102da0 0x7f5dec1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 -- 172.21.15.31:0/981109792 >> 172.21.15.31:0/981109792 conn(0x7f5dec0fb370 msgr2=0x7f5dec0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 -- 172.21.15.31:0/981109792 shutdown_connections 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 -- 172.21.15.31:0/981109792 wait complete. 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.157+0000 7f5df2112640 1 Processor -- start 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 -- start start 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec1001d0 0x7f5dec197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dec102da0 0x7f5dec19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec10d2d0 con 0x7f5dec0ff810 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec10d150 con 0x7f5dec1001d0 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5df2112640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec10d450 con 0x7f5dec102da0 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53572/0 (socket says 172.21.15.31:53572) 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 -- 172.21.15.31:0/2011781616 learned_addr learned my addr 172.21.15.31:0/2011781616 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deaffd640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec1001d0 0x7f5dec197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5debfff640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dec102da0 0x7f5dec19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 -- 172.21.15.31:0/2011781616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dec102da0 msgr2=0x7f5dec19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dec102da0 0x7f5dec19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 -- 172.21.15.31:0/2011781616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec1001d0 msgr2=0x7f5dec197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec1001d0 0x7f5dec197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.158+0000 7f5deb7fe640 1 -- 172.21.15.31:0/2011781616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19f5c0 con 0x7f5dec0ff810 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.159+0000 7f5deb7fe640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec1979a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00d950 tx=0x7f5ddc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.159+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc017070 con 0x7f5dec0ff810 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.159+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19f8b0 con 0x7f5dec0ff810 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.159+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc009c30 con 0x7f5dec0ff810 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.159+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc00f620 con 0x7f5dec0ff810 2024-04-25T00:42:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.159+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec19fdd0 con 0x7f5dec0ff810 2024-04-25T00:42:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.160+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec108ce0 con 0x7f5dec0ff810 2024-04-25T00:42:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.161+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5ddc01a070 con 0x7f5dec0ff810 2024-04-25T00:42:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.161+0000 7f5de8ff9640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5dd40a2b90 0x7f5dd40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.162+0000 7f5deaffd640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5dd40a2b90 0x7f5dd40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.162+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f5ddc0cead0 con 0x7f5dec0ff810 2024-04-25T00:42:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.162+0000 7f5deaffd640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5dd40a2b90 0x7f5dd40a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5de0006010 tx=0x7f5de0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.166+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ddc009de0 con 0x7f5dec0ff810 2024-04-25T00:42:34.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.351+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5dec062340 con 0x7f5dec0ff810 2024-04-25T00:42:34.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.352+0000 7f5de8ff9640 1 -- 172.21.15.31:0/2011781616 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f5ddc09ba00 con 0x7f5dec0ff810 2024-04-25T00:42:34.352 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5dd40a2b90 msgr2=0x7f5dd40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5dd40a2b90 0x7f5dd40a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5de0006010 tx=0x7f5de0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 msgr2=0x7f5dec1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec1979a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00d950 tx=0x7f5ddc00de10 comp rx=0 tx=0).stop 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 shutdown_connections 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dec102da0 0x7f5dec19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec1001d0 0x7f5dec197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f5dd40a2b90 0x7f5dd40a5050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 --2- 172.21.15.31:0/2011781616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dec0ff810 0x7f5dec1979a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 >> 172.21.15.31:0/2011781616 conn(0x7f5dec0fb370 msgr2=0x7f5dec105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 shutdown_connections 2024-04-25T00:42:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.354+0000 7f5df2112640 1 -- 172.21.15.31:0/2011781616 wait complete. 2024-04-25T00:42:34.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 -- 172.21.15.31:0/504281738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac101780 msgr2=0x7f2bac101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/504281738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac101780 0x7f2bac101ba0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009900 tx=0x7f2b9c02a410 comp rx=0 tx=0).stop 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 -- 172.21.15.31:0/504281738 shutdown_connections 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/504281738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac103330 0x7f2bac109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/504281738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bac102970 0x7f2bac102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/504281738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac101780 0x7f2bac101ba0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 -- 172.21.15.31:0/504281738 >> 172.21.15.31:0/504281738 conn(0x7f2bac0fccf0 msgr2=0x7f2bac0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 -- 172.21.15.31:0/504281738 shutdown_connections 2024-04-25T00:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 -- 172.21.15.31:0/504281738 wait complete. 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.471+0000 7f2bb12bd640 1 Processor -- start 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 -- start start 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bac101780 0x7f2bac0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac102970 0x7f2bac073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 0x7f2bac074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10d2a0 con 0x7f2bac103330 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10d120 con 0x7f2bac102970 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bb12bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10d420 con 0x7f2bac101780 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 0x7f2bac074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 0x7f2bac074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53592/0 (socket says 172.21.15.31:53592) 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 -- 172.21.15.31:0/223327440 learned_addr learned my addr 172.21.15.31:0/223327440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2baa7fc640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac102970 0x7f2bac073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2baaffd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bac101780 0x7f2bac0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 -- 172.21.15.31:0/223327440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bac101780 msgr2=0x7f2bac0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bac101780 0x7f2bac0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 -- 172.21.15.31:0/223327440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac102970 msgr2=0x7f2bac073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac102970 0x7f2bac073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 -- 172.21.15.31:0/223327440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac074900 con 0x7f2bac103330 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2bab7fe640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 0x7f2bac074070 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400eb10 tx=0x7f2ba400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.472+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4004b20 con 0x7f2bac103330 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.473+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac1a1b50 con 0x7f2bac103330 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.473+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba4004cc0 con 0x7f2bac103330 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.473+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4010690 con 0x7f2bac103330 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.473+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac1a2050 con 0x7f2bac103330 2024-04-25T00:42:34.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.474+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac108c10 con 0x7f2bac103330 2024-04-25T00:42:34.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.475+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ba4005020 con 0x7f2bac103330 2024-04-25T00:42:34.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.475+0000 7f2b93fff640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2b940a2b90 0x7f2b940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.475+0000 7f2baaffd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2b940a2b90 0x7f2b940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.475+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2ba40cf720 con 0x7f2bac103330 2024-04-25T00:42:34.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.476+0000 7f2baaffd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2b940a2b90 0x7f2b940a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c000c00 tx=0x7f2b9c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.480+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ba409c650 con 0x7f2bac103330 2024-04-25T00:42:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.667+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2bac0623e0 con 0x7f2bac103330 2024-04-25T00:42:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.667+0000 7f2b93fff640 1 -- 172.21.15.31:0/223327440 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f2ba409bf50 con 0x7f2bac103330 2024-04-25T00:42:34.668 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T00:42:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2b940a2b90 msgr2=0x7f2b940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2b940a2b90 0x7f2b940a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c000c00 tx=0x7f2b9c0314a0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 msgr2=0x7f2bac074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 0x7f2bac074070 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400eb10 tx=0x7f2ba400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 shutdown_connections 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f2b940a2b90 0x7f2b940a5050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bac103330 0x7f2bac074070 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac102970 0x7f2bac073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 --2- 172.21.15.31:0/223327440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bac101780 0x7f2bac0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.669+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 >> 172.21.15.31:0/223327440 conn(0x7f2bac0fccf0 msgr2=0x7f2bac105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.670+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 shutdown_connections 2024-04-25T00:42:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.670+0000 7f2bb12bd640 1 -- 172.21.15.31:0/223327440 wait complete. 2024-04-25T00:42:34.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:42:34.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 -- 172.21.15.31:0/3650675251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8638101890 msgr2=0x7f8638101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3650675251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8638101890 0x7f8638101d10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f862c009900 tx=0x7f862c02a410 comp rx=0 tx=0).stop 2024-04-25T00:42:34.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 -- 172.21.15.31:0/3650675251 shutdown_connections 2024-04-25T00:42:34.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3650675251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638102250 0x7f8638108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3650675251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8638101890 0x7f8638101d10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3650675251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86381006a0 0x7f8638100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 -- 172.21.15.31:0/3650675251 >> 172.21.15.31:0/3650675251 conn(0x7f86380fbc30 msgr2=0x7f86380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 -- 172.21.15.31:0/3650675251 shutdown_connections 2024-04-25T00:42:34.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.785+0000 7f863c9e7640 1 -- 172.21.15.31:0/3650675251 wait complete. 2024-04-25T00:42:34.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 Processor -- start 2024-04-25T00:42:34.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 -- start start 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 0x7f86381968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638101890 0x7f8638196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638102250 0x7f863819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f863810c1b0 con 0x7f86381006a0 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f863810c030 con 0x7f8638101890 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863c9e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f863810c330 con 0x7f8638102250 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 0x7f86381968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 0x7f86381968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53608/0 (socket says 172.21.15.31:53608) 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863659c640 1 -- 172.21.15.31:0/3758264219 learned_addr learned my addr 172.21.15.31:0/3758264219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f8635d9b640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638101890 0x7f8638196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f8636d9d640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638102250 0x7f863819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.786+0000 7f863659c640 1 -- 172.21.15.31:0/3758264219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638102250 msgr2=0x7f863819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863659c640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638102250 0x7f863819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863659c640 1 -- 172.21.15.31:0/3758264219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638101890 msgr2=0x7f8638196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863659c640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638101890 0x7f8638196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863659c640 1 -- 172.21.15.31:0/3758264219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863819e4e0 con 0x7f86381006a0 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f8636d9d640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638102250 0x7f863819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f8635d9b640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638101890 0x7f8638196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863659c640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 0x7f86381968c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f862800d770 tx=0x7f862800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628017070 con 0x7f86381006a0 2024-04-25T00:42:34.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8628009c30 con 0x7f86381006a0 2024-04-25T00:42:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628003d00 con 0x7f86381006a0 2024-04-25T00:42:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863819e7d0 con 0x7f86381006a0 2024-04-25T00:42:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.787+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863819eca0 con 0x7f86381006a0 2024-04-25T00:42:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.788+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8638107ab0 con 0x7f86381006a0 2024-04-25T00:42:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.789+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8628003ea0 con 0x7f86381006a0 2024-04-25T00:42:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.790+0000 7f86277fe640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f86200a2b90 0x7f86200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.790+0000 7f8635d9b640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f86200a2b90 0x7f86200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.790+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f86280cedb0 con 0x7f86381006a0 2024-04-25T00:42:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.790+0000 7f8635d9b640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f86200a2b90 0x7f86200a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f862c006010 tx=0x7f862c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.793+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f862809bce0 con 0x7f86381006a0 2024-04-25T00:42:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:34 smithi169 ceph-mon[33112]: pgmap v721: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:42:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2604112604' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:42:34.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1659936113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:34.890 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:34 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2011781616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:34 smithi040 ceph-mon[32945]: pgmap v721: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:42:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2604112604' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:42:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1659936113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:34.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:34 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2011781616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.977+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f863802d320 con 0x7f86381006a0 2024-04-25T00:42:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.977+0000 7f86277fe640 1 -- 172.21.15.31:0/3758264219 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f8628009de0 con 0x7f86381006a0 2024-04-25T00:42:34.979 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T00:42:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f86200a2b90 msgr2=0x7f86200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f86200a2b90 0x7f86200a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f862c006010 tx=0x7f862c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 msgr2=0x7f86381968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 0x7f86381968c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f862800d770 tx=0x7f862800dc30 comp rx=0 tx=0).stop 2024-04-25T00:42:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 shutdown_connections 2024-04-25T00:42:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638102250 0x7f863819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8638101890 0x7f8638196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f86200a2b90 0x7f86200a5050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 --2- 172.21.15.31:0/3758264219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86381006a0 0x7f86381968c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 >> 172.21.15.31:0/3758264219 conn(0x7f86380fbc30 msgr2=0x7f8638104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 shutdown_connections 2024-04-25T00:42:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:34.979+0000 7f863c9e7640 1 -- 172.21.15.31:0/3758264219 wait complete. 2024-04-25T00:42:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:34 smithi031 ceph-mon[29167]: pgmap v721: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:42:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2604112604' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:42:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1659936113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:34 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2011781616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:42:34.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:42:35.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 -- 172.21.15.31:0/2916446358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 msgr2=0x7fe1a8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/2916446358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a8108ae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009ca0 tx=0x7fe1a002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 -- 172.21.15.31:0/2916446358 shutdown_connections 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/2916446358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a8108ae0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/2916446358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a8101890 0x7fe1a8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/2916446358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a81006a0 0x7fe1a8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 -- 172.21.15.31:0/2916446358 >> 172.21.15.31:0/2916446358 conn(0x7fe1a80fbc30 msgr2=0x7fe1a80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 -- 172.21.15.31:0/2916446358 shutdown_connections 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.093+0000 7fe1afd1b640 1 -- 172.21.15.31:0/2916446358 wait complete. 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 Processor -- start 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 -- start start 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a81006a0 0x7fe1a8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a8101890 0x7fe1a8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a810d350 con 0x7fe1a8102250 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a810d1d0 con 0x7fe1a8101890 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1afd1b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a810d4d0 con 0x7fe1a81006a0 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53618/0 (socket says 172.21.15.31:53618) 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 -- 172.21.15.31:0/3199558613 learned_addr learned my addr 172.21.15.31:0/3199558613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1adab7640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a81006a0 0x7fe1a8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ad2b6640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a8101890 0x7fe1a8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 -- 172.21.15.31:0/3199558613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a81006a0 msgr2=0x7fe1a8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a81006a0 0x7fe1a8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 -- 172.21.15.31:0/3199558613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a8101890 msgr2=0x7fe1a8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a8101890 0x7fe1a8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 -- 172.21.15.31:0/3199558613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a819f5b0 con 0x7fe1a8102250 2024-04-25T00:42:35.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe1ae2b8640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a819ef40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0031040 tx=0x7fe1a0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.094+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0032ec0 con 0x7fe1a8102250 2024-04-25T00:42:35.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.095+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a00044f0 con 0x7fe1a8102250 2024-04-25T00:42:35.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.095+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a819f840 con 0x7fe1a8102250 2024-04-25T00:42:35.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.095+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0030cb0 con 0x7fe1a8102250 2024-04-25T00:42:35.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.095+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a819fd90 con 0x7fe1a8102250 2024-04-25T00:42:35.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.096+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a8107a90 con 0x7fe1a8102250 2024-04-25T00:42:35.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.097+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe1a0004040 con 0x7fe1a8102250 2024-04-25T00:42:35.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.097+0000 7fe19effd640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe1940a2d30 0x7fe1940a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.098+0000 7fe1adab7640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe1940a2d30 0x7fe1940a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.098+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe1a00f0620 con 0x7fe1a8102250 2024-04-25T00:42:35.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.098+0000 7fe1adab7640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe1940a2d30 0x7fe1940a51f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe1a81016f0 tx=0x7fe1a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.102+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1a00bd550 con 0x7fe1a8102250 2024-04-25T00:42:35.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.301+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe1a8062340 con 0x7fe1a8102250 2024-04-25T00:42:35.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.302+0000 7fe19effd640 1 -- 172.21.15.31:0/3199558613 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+592 (secure 0 0 0) 0x7fe1a00bce50 con 0x7fe1a8102250 2024-04-25T00:42:35.303 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3395718020 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:35.303 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3443312222 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3443312222 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3607070745 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3134587413 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3135101963 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1911777180 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3278889828 2024-04-26T00:21:13.218737+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1911777180 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4126189511 2024-04-26T00:20:45.382327+0000 2024-04-25T00:42:35.304 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2024-04-25T00:42:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe1940a2d30 msgr2=0x7fe1940a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe1940a2d30 0x7fe1940a51f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe1a81016f0 tx=0x7fe1a4005f90 comp rx=0 tx=0).stop 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 msgr2=0x7fe1a819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a819ef40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0031040 tx=0x7fe1a0003ba0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 shutdown_connections 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fe1940a2d30 0x7fe1940a51f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a8102250 0x7fe1a819ef40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a8101890 0x7fe1a8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 --2- 172.21.15.31:0/3199558613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a81006a0 0x7fe1a8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 >> 172.21.15.31:0/3199558613 conn(0x7fe1a80fbc30 msgr2=0x7fe1a8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 shutdown_connections 2024-04-25T00:42:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:35.304+0000 7fe1afd1b640 1 -- 172.21.15.31:0/3199558613 wait complete. 2024-04-25T00:42:35.316 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-04-25T00:42:35.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:42:35.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:42:35.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[78687]: fuse finished with error 0 and tester_r 0 2024-04-25T00:42:35.342 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:35 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/223327440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:42:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:35 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3758264219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:42:35.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:35 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3199558613' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:42:36.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:35 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/223327440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:42:36.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:35 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3758264219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:42:36.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:35 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3199558613' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:42:36.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:35 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/223327440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:42:36.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:35 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3758264219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:42:36.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:35 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3199558613' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:42:36.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:36 smithi040 ceph-mon[32945]: pgmap v722: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:37.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:36 smithi031 ceph-mon[29167]: pgmap v722: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:37.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:36 smithi169 ceph-mon[33112]: pgmap v722: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:42:38.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:38 smithi040 ceph-mon[32945]: pgmap v723: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:39.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:38 smithi031 ceph-mon[29167]: pgmap v723: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:39.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:38 smithi169 ceph-mon[33112]: pgmap v723: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T00:42:41.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:40 smithi031 ceph-mon[29167]: pgmap v724: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-25T00:42:41.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:40 smithi169 ceph-mon[33112]: pgmap v724: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-25T00:42:41.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:40 smithi040 ceph-mon[32945]: pgmap v724: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-25T00:42:41.443 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:42:41.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:42:41.444 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:42:41.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:42:41.474 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T00:42:41.532 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:42:41.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:42:41.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:42:41.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:42:41.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-25T00:42:41.703 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T00:42:41.714 INFO:tasks.cephadm:Teardown begin 2024-04-25T00:42:41.714 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:42:41.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 -- 172.21.15.31:0/4270409864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 msgr2=0x7f4f18100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 --2- 172.21.15.31:0/4270409864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18100ac0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009900 tx=0x7f4f1402a410 comp rx=0 tx=0).stop 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 -- 172.21.15.31:0/4270409864 shutdown_connections 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 --2- 172.21.15.31:0/4270409864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 0x7f4f18108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 --2- 172.21.15.31:0/4270409864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f18101890 0x7f4f18101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 --2- 172.21.15.31:0/4270409864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18100ac0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 -- 172.21.15.31:0/4270409864 >> 172.21.15.31:0/4270409864 conn(0x7f4f180fbc30 msgr2=0x7f4f180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 -- 172.21.15.31:0/4270409864 shutdown_connections 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 -- 172.21.15.31:0/4270409864 wait complete. 2024-04-25T00:42:41.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 Processor -- start 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.822+0000 7f4f20415640 1 -- start start 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f18101890 0x7f4f18197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 0x7f4f1819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810c220 con 0x7f4f181006a0 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810c0a0 con 0x7f4f18102250 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810c3a0 con 0x7f4f18101890 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53624/0 (socket says 172.21.15.31:53624) 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 -- 172.21.15.31:0/1843589364 learned_addr learned my addr 172.21.15.31:0/1843589364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1d9b0640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f18101890 0x7f4f18197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e9b2640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 0x7f4f1819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 -- 172.21.15.31:0/1843589364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f18101890 msgr2=0x7f4f18197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f18101890 0x7f4f18197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 -- 172.21.15.31:0/1843589364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 msgr2=0x7f4f1819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 0x7f4f1819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 -- 172.21.15.31:0/1843589364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1819f670 con 0x7f4f181006a0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e9b2640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 0x7f4f1819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f1e1b1640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18197a50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4f140314e0 tx=0x7f4f140042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1403f030 con 0x7f4f181006a0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f14003cb0 con 0x7f4f181006a0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14035070 con 0x7f4f181006a0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.823+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1819f900 con 0x7f4f181006a0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.824+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1819fec0 con 0x7f4f181006a0 2024-04-25T00:42:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.824+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f181a0090 con 0x7f4f181006a0 2024-04-25T00:42:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.826+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4f14042030 con 0x7f4f181006a0 2024-04-25T00:42:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.826+0000 7f4f0f7fe640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4f040a2c10 0x7f4f040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.827+0000 7f4f1d9b0640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4f040a2c10 0x7f4f040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.827+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4f140ef620 con 0x7f4f181006a0 2024-04-25T00:42:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.827+0000 7f4f1d9b0640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4f040a2c10 0x7f4f040a50d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4f18198f60 tx=0x7f4f0800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:41.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:41.831+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f140bc550 con 0x7f4f181006a0 2024-04-25T00:42:42.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.057+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f181a0260 con 0x7f4f181006a0 2024-04-25T00:42:42.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.057+0000 7f4f0f7fe640 1 -- 172.21.15.31:0/1843589364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11926 (secure 0 0 0) 0x7f4f140bbe50 con 0x7f4f181006a0 2024-04-25T00:42:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:42:42.058 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:30:28.982863+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24424,24434,24499,24512,24517]},"id":1}]} 2024-04-25T00:42:42.058 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4f040a2c10 msgr2=0x7f4f040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4f040a2c10 0x7f4f040a50d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4f18198f60 tx=0x7f4f0800b390 comp rx=0 tx=0).stop 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 msgr2=0x7f4f18197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18197a50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4f140314e0 tx=0x7f4f140042e0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 shutdown_connections 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f18102250 0x7f4f1819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f18101890 0x7f4f18197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f4f040a2c10 0x7f4f040a50d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 --2- 172.21.15.31:0/1843589364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f181006a0 0x7f4f18197a50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.059+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 >> 172.21.15.31:0/1843589364 conn(0x7f4f180fbc30 msgr2=0x7f4f18069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:42.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.060+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 shutdown_connections 2024-04-25T00:42:42.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.060+0000 7f4f20415640 1 -- 172.21.15.31:0/1843589364 wait complete. 2024-04-25T00:42:42.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 -- 172.21.15.31:0/3995110810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec101790 msgr2=0x7f46ec101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 --2- 172.21.15.31:0/3995110810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec101790 0x7f46ec101bb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009900 tx=0x7f46dc02a410 comp rx=0 tx=0).stop 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 -- 172.21.15.31:0/3995110810 shutdown_connections 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 --2- 172.21.15.31:0/3995110810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46ec103340 0x7f46ec109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 --2- 172.21.15.31:0/3995110810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec102980 0x7f46ec102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 --2- 172.21.15.31:0/3995110810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec101790 0x7f46ec101bb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 -- 172.21.15.31:0/3995110810 >> 172.21.15.31:0/3995110810 conn(0x7f46ec0fcd20 msgr2=0x7f46ec0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 -- 172.21.15.31:0/3995110810 shutdown_connections 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 -- 172.21.15.31:0/3995110810 wait complete. 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 Processor -- start 2024-04-25T00:42:42.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.173+0000 7f46f2742640 1 -- start start 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f2742640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46ec101790 0x7f46ec0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f2742640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 0x7f46ec073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f2742640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec103340 0x7f46ec074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f2742640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10d2c0 con 0x7f46ec102980 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f2742640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10d140 con 0x7f46ec103340 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f2742640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10d440 con 0x7f46ec101790 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 0x7f46ec073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 0x7f46ec073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53640/0 (socket says 172.21.15.31:53640) 2024-04-25T00:42:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 -- 172.21.15.31:0/2660425048 learned_addr learned my addr 172.21.15.31:0/2660425048 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46ebfff640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46ec101790 0x7f46ec0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f0cdf640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec103340 0x7f46ec074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 -- 172.21.15.31:0/2660425048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46ec101790 msgr2=0x7f46ec0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46ec101790 0x7f46ec0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 -- 172.21.15.31:0/2660425048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec103340 msgr2=0x7f46ec074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec103340 0x7f46ec074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 -- 172.21.15.31:0/2660425048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec074900 con 0x7f46ec102980 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46f0cdf640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec103340 0x7f46ec074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.174+0000 7f46eb7fe640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 0x7f46ec073b30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f46e000d950 tx=0x7f46e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.175+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0017070 con 0x7f46ec102980 2024-04-25T00:42:42.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.175+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1a1ae0 con 0x7f46ec102980 2024-04-25T00:42:42.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.175+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46e0009c30 con 0x7f46ec102980 2024-04-25T00:42:42.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.175+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000f620 con 0x7f46ec102980 2024-04-25T00:42:42.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.175+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a2080 con 0x7f46ec102980 2024-04-25T00:42:42.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.176+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec108b00 con 0x7f46ec102980 2024-04-25T00:42:42.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.177+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f46e00109a0 con 0x7f46ec102980 2024-04-25T00:42:42.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.178+0000 7f46e97fa640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f46d40a2b90 0x7f46d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.178+0000 7f46ebfff640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f46d40a2b90 0x7f46d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.178+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f46e00cedb0 con 0x7f46ec102980 2024-04-25T00:42:42.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.178+0000 7f46ebfff640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f46d40a2b90 0x7f46d40a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46dc000c00 tx=0x7f46dc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.183+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46e00d3070 con 0x7f46ec102980 2024-04-25T00:42:42.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.406+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46ec062340 con 0x7f46ec102980 2024-04-25T00:42:42.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.407+0000 7f46e97fa640 1 -- 172.21.15.31:0/2660425048 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11926 (secure 0 0 0) 0x7f46e0009de0 con 0x7f46ec102980 2024-04-25T00:42:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:42:42.408 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/951372407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":951372407},{"type":"v1","addr":"172.21.15.31:6837","nonce":951372407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2151696509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2151696509},{"type":"v1","addr":"172.21.15.31:6839","nonce":2151696509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:28:23.753095+0000","modified":"2024-04-25T00:30:28.982863+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24499,"mds_3":24434,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2045809592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2045809592},{"type":"v1","addr":"172.21.15.31:6841","nonce":2045809592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6833/3504624607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3504624607},{"type":"v1","addr":"172.21.15.40:6833","nonce":3504624607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6833/2672070962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2672070962},{"type":"v1","addr":"172.21.15.169:6833","nonce":2672070962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"j","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.31:6835/310214957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":310214957},{"type":"v1","addr":"172.21.15.31:6835","nonce":310214957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/653343360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":653343360},{"type":"v1","addr":"172.21.15.40:6835","nonce":653343360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3765244618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3765244618},{"type":"v1","addr":"172.21.15.40:6837","nonce":3765244618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3258138877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3258138877},{"type":"v1","addr":"172.21.15.169:6835","nonce":3258138877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6837/1684471092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1684471092},{"type":"v1","addr":"172.21.15.169:6837","nonce":1684471092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6839/2724938586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":2724938586},{"type":"v1","addr":"172.21.15.40:6839","nonce":2724938586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/3109017383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":3109017383},{"type":"v1","addr":"172.21.15.169:6839","nonce":3109017383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24424,24434,24499,24512,24517]},"id":1}]} 2024-04-25T00:42:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f46d40a2b90 msgr2=0x7f46d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f46d40a2b90 0x7f46d40a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46dc000c00 tx=0x7f46dc0314a0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 msgr2=0x7f46ec073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 0x7f46ec073b30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f46e000d950 tx=0x7f46e000de10 comp rx=0 tx=0).stop 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 shutdown_connections 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46ec103340 0x7f46ec074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f46d40a2b90 0x7f46d40a5050 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46ec102980 0x7f46ec073b30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 --2- 172.21.15.31:0/2660425048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46ec101790 0x7f46ec0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 >> 172.21.15.31:0/2660425048 conn(0x7f46ec0fcd20 msgr2=0x7f46ec105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 shutdown_connections 2024-04-25T00:42:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.409+0000 7f46f2742640 1 -- 172.21.15.31:0/2660425048 wait complete. 2024-04-25T00:42:42.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.522+0000 7fee6cc9a640 1 -- 172.21.15.31:0/723019232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68101890 msgr2=0x7fee68101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.522+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/723019232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68101890 0x7fee68101d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009900 tx=0x7fee5c02a410 comp rx=0 tx=0).stop 2024-04-25T00:42:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 -- 172.21.15.31:0/723019232 shutdown_connections 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/723019232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee68102250 0x7fee68108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/723019232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68101890 0x7fee68101d10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/723019232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee681006a0 0x7fee68100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 -- 172.21.15.31:0/723019232 >> 172.21.15.31:0/723019232 conn(0x7fee680fbc30 msgr2=0x7fee680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 -- 172.21.15.31:0/723019232 shutdown_connections 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 -- 172.21.15.31:0/723019232 wait complete. 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 Processor -- start 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.523+0000 7fee6cc9a640 1 -- start start 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681006a0 0x7fee6810b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee68101890 0x7fee68109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 0x7fee68109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee6810d2f0 con 0x7fee68102250 2024-04-25T00:42:42.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee6810d170 con 0x7fee681006a0 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee6810d470 con 0x7fee68101890 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 0x7fee68109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 0x7fee68109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53656/0 (socket says 172.21.15.31:53656) 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 -- 172.21.15.31:0/2132365707 learned_addr learned my addr 172.21.15.31:0/2132365707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee65d9b640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee68101890 0x7fee68109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6659c640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681006a0 0x7fee6810b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 -- 172.21.15.31:0/2132365707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee68101890 msgr2=0x7fee68109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee68101890 0x7fee68109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 -- 172.21.15.31:0/2132365707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681006a0 msgr2=0x7fee6810b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681006a0 0x7fee6810b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 -- 172.21.15.31:0/2132365707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6810a430 con 0x7fee68102250 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6659c640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681006a0 0x7fee6810b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee66d9d640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 0x7fee68109ba0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee6000eaa0 tx=0x7fee6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6000cdb0 con 0x7fee68102250 2024-04-25T00:42:42.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee60005020 con 0x7fee68102250 2024-04-25T00:42:42.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee681a1ac0 con 0x7fee68102250 2024-04-25T00:42:42.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee600116d0 con 0x7fee68102250 2024-04-25T00:42:42.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.524+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee681a1fe0 con 0x7fee68102250 2024-04-25T00:42:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.526+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee680689b0 con 0x7fee68102250 2024-04-25T00:42:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.527+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fee600051d0 con 0x7fee68102250 2024-04-25T00:42:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.527+0000 7fee577fe640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fee500a2c10 0x7fee500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.527+0000 7fee6659c640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fee500a2c10 0x7fee500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.527+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fee600cf240 con 0x7fee68102250 2024-04-25T00:42:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.528+0000 7fee6659c640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fee500a2c10 0x7fee500a50d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee681016f0 tx=0x7fee58005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:42.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.532+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee60016030 con 0x7fee68102250 2024-04-25T00:42:42.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.719+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee68062400 con 0x7fee68102250 2024-04-25T00:42:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.720+0000 7fee577fe640 1 -- 172.21.15.31:0/2132365707 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22698 (secure 0 0 0) 0x7fee6009c170 con 0x7fee68102250 2024-04-25T00:42:42.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:42:42.721 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"6c091b24-0299-11ef-bc93-c7b262605968","created":"2024-04-25T00:20:13.144267+0000","modified":"2024-04-25T00:30:15.394985+0000","last_up_change":"2024-04-25T00:27:33.521764+0000","last_in_change":"2024-04-25T00:27:13.318774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:23:41.749279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:28:21.908076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:28:23.150234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5605681-c8ce-4528-a557-32de873bf2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6803","nonce":2938713990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6805","nonce":2938713990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6809","nonce":2938713990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2938713990},{"type":"v1","addr":"172.21.15.31:6807","nonce":2938713990}]},"public_addr":"172.21.15.31:6803/2938713990","cluster_addr":"172.21.15.31:6805/2938713990","heartbeat_back_addr":"172.21.15.31:6809/2938713990","heartbeat_front_addr":"172.21.15.31:6807/2938713990","state":["exists","up"]},{"osd":1,"uuid":"32f950d6-4574-449b-8273-82833bc7774d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6801","nonce":2453389189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6803","nonce":2453389189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6807","nonce":2453389189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2453389189},{"type":"v1","addr":"172.21.15.40:6805","nonce":2453389189}]},"public_addr":"172.21.15.40:6801/2453389189","cluster_addr":"172.21.15.40:6803/2453389189","heartbeat_back_addr":"172.21.15.40:6807/2453389189","heartbeat_front_addr":"172.21.15.40:6805/2453389189","state":["exists","up"]},{"osd":2,"uuid":"1cc669bc-41dc-4b6a-b5ed-b8e633e56a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6801","nonce":1729358073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6803","nonce":1729358073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6807","nonce":1729358073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1729358073},{"type":"v1","addr":"172.21.15.169:6805","nonce":1729358073}]},"public_addr":"172.21.15.169:6801/1729358073","cluster_addr":"172.21.15.169:6803/1729358073","heartbeat_back_addr":"172.21.15.169:6807/1729358073","heartbeat_front_addr":"172.21.15.169:6805/1729358073","state":["exists","up"]},{"osd":3,"uuid":"938ed94c-292f-4151-ad75-7a901afb1717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6811","nonce":2474264832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6813","nonce":2474264832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6817","nonce":2474264832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2474264832},{"type":"v1","addr":"172.21.15.31:6815","nonce":2474264832}]},"public_addr":"172.21.15.31:6811/2474264832","cluster_addr":"172.21.15.31:6813/2474264832","heartbeat_back_addr":"172.21.15.31:6817/2474264832","heartbeat_front_addr":"172.21.15.31:6815/2474264832","state":["exists","up"]},{"osd":4,"uuid":"d9d02417-522a-472c-bc86-eb4a65662c6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6809","nonce":3167270680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6811","nonce":3167270680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6815","nonce":3167270680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3167270680},{"type":"v1","addr":"172.21.15.40:6813","nonce":3167270680}]},"public_addr":"172.21.15.40:6809/3167270680","cluster_addr":"172.21.15.40:6811/3167270680","heartbeat_back_addr":"172.21.15.40:6815/3167270680","heartbeat_front_addr":"172.21.15.40:6813/3167270680","state":["exists","up"]},{"osd":5,"uuid":"c51c2807-c67a-46ef-8add-0ee8009cdd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6809","nonce":4050548275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6811","nonce":4050548275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6815","nonce":4050548275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":4050548275},{"type":"v1","addr":"172.21.15.169:6813","nonce":4050548275}]},"public_addr":"172.21.15.169:6809/4050548275","cluster_addr":"172.21.15.169:6811/4050548275","heartbeat_back_addr":"172.21.15.169:6815/4050548275","heartbeat_front_addr":"172.21.15.169:6813/4050548275","state":["exists","up"]},{"osd":6,"uuid":"c4034099-a0bd-4ef2-95a1-3b00255b587e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6819","nonce":3697284546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6821","nonce":3697284546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6825","nonce":3697284546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3697284546},{"type":"v1","addr":"172.21.15.31:6823","nonce":3697284546}]},"public_addr":"172.21.15.31:6819/3697284546","cluster_addr":"172.21.15.31:6821/3697284546","heartbeat_back_addr":"172.21.15.31:6825/3697284546","heartbeat_front_addr":"172.21.15.31:6823/3697284546","state":["exists","up"]},{"osd":7,"uuid":"cc3fde10-5c71-4d9b-a4b0-b184780d5286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6817","nonce":4053415057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6819","nonce":4053415057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6823","nonce":4053415057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":4053415057},{"type":"v1","addr":"172.21.15.40:6821","nonce":4053415057}]},"public_addr":"172.21.15.40:6817/4053415057","cluster_addr":"172.21.15.40:6819/4053415057","heartbeat_back_addr":"172.21.15.40:6823/4053415057","heartbeat_front_addr":"172.21.15.40:6821/4053415057","state":["exists","up"]},{"osd":8,"uuid":"ddad000a-a81c-4a3d-b5ef-ca98e6a57bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6817","nonce":3220692863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6819","nonce":3220692863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6823","nonce":3220692863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3220692863},{"type":"v1","addr":"172.21.15.169:6821","nonce":3220692863}]},"public_addr":"172.21.15.169:6817/3220692863","cluster_addr":"172.21.15.169:6819/3220692863","heartbeat_back_addr":"172.21.15.169:6823/3220692863","heartbeat_front_addr":"172.21.15.169:6821/3220692863","state":["exists","up"]},{"osd":9,"uuid":"d854d1bd-461f-4def-b383-20908701640b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6827","nonce":1961015503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6829","nonce":1961015503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6833","nonce":1961015503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1961015503},{"type":"v1","addr":"172.21.15.31:6831","nonce":1961015503}]},"public_addr":"172.21.15.31:6827/1961015503","cluster_addr":"172.21.15.31:6829/1961015503","heartbeat_back_addr":"172.21.15.31:6833/1961015503","heartbeat_front_addr":"172.21.15.31:6831/1961015503","state":["exists","up"]},{"osd":10,"uuid":"413c2a5c-f9a1-4a20-838d-624a8afae359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6825","nonce":3974437906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6827","nonce":3974437906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6831","nonce":3974437906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3974437906},{"type":"v1","addr":"172.21.15.40:6829","nonce":3974437906}]},"public_addr":"172.21.15.40:6825/3974437906","cluster_addr":"172.21.15.40:6827/3974437906","heartbeat_back_addr":"172.21.15.40:6831/3974437906","heartbeat_front_addr":"172.21.15.40:6829/3974437906","state":["exists","up"]},{"osd":11,"uuid":"0d81d667-30bf-47da-824f-51eb05444477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6825","nonce":3987371467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6827","nonce":3987371467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6831","nonce":3987371467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3987371467},{"type":"v1","addr":"172.21.15.169:6829","nonce":3987371467}]},"public_addr":"172.21.15.169:6825/3987371467","cluster_addr":"172.21.15.169:6827/3987371467","heartbeat_back_addr":"172.21.15.169:6831/3987371467","heartbeat_front_addr":"172.21.15.169:6829/3987371467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:22:50.914714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:14.777643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:23:38.493504+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:04.647696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:30.154200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:24:54.924768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:21.215107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:25:46.940134+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:11.803736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:26:39.524270+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:05.444055+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:27:31.940423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3395718020":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/3443312222":"2024-04-26T00:21:13.218737+0000","172.21.15.31:0/3607070745":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3134587413":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3135101963":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6800/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/3278889828":"2024-04-26T00:21:13.218737+0000","172.21.15.31:6801/1911777180":"2024-04-26T00:20:45.382327+0000","172.21.15.31:0/4126189511":"2024-04-26T00:20:45.382327+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fee500a2c10 msgr2=0x7fee500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fee500a2c10 0x7fee500a50d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee681016f0 tx=0x7fee58005e70 comp rx=0 tx=0).stop 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 msgr2=0x7fee68109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 0x7fee68109ba0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee6000eaa0 tx=0x7fee6000ef60 comp rx=0 tx=0).stop 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 shutdown_connections 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fee500a2c10 0x7fee500a50d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee68102250 0x7fee68109ba0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee68101890 0x7fee68109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 --2- 172.21.15.31:0/2132365707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681006a0 0x7fee6810b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 >> 172.21.15.31:0/2132365707 conn(0x7fee680fbc30 msgr2=0x7fee68104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 shutdown_connections 2024-04-25T00:42:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.722+0000 7fee6cc9a640 1 -- 172.21.15.31:0/2132365707 wait complete. 2024-04-25T00:42:42.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:42 smithi031 ceph-mon[29167]: pgmap v725: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T00:42:42.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:42 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1843589364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:42:42.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:42 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2660425048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:42:42.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:42:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 -- 172.21.15.31:0/3836070839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8101890 msgr2=0x7f8ac8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/3836070839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8101890 0x7f8ac8101d10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009900 tx=0x7f8ab002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 -- 172.21.15.31:0/3836070839 shutdown_connections 2024-04-25T00:42:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/3836070839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8102250 0x7f8ac8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/3836070839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8101890 0x7f8ac8101d10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/3836070839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac81006a0 0x7f8ac8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 -- 172.21.15.31:0/3836070839 >> 172.21.15.31:0/3836070839 conn(0x7f8ac80fbc30 msgr2=0x7f8ac80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 -- 172.21.15.31:0/3836070839 shutdown_connections 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.837+0000 7f8acd3ee640 1 -- 172.21.15.31:0/3836070839 wait complete. 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 Processor -- start 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 -- start start 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 0x7f8ac81969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac8101890 0x7f8ac8196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8102250 0x7f8ac819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810c200 con 0x7f8ac81006a0 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810c080 con 0x7f8ac8102250 2024-04-25T00:42:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8acd3ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810c380 con 0x7f8ac8101890 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 0x7f8ac81969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 0x7f8ac81969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49044/0 (socket says 172.21.15.31:49044) 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 -- 172.21.15.31:0/4195985255 learned_addr learned my addr 172.21.15.31:0/4195985255 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8abffff640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac8101890 0x7f8ac8196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 -- 172.21.15.31:0/4195985255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac8101890 msgr2=0x7f8ac8196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac8101890 0x7f8ac8196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac77fe640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8102250 0x7f8ac819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 -- 172.21.15.31:0/4195985255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8102250 msgr2=0x7f8ac819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8102250 0x7f8ac819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8ac6ffd640 1 -- 172.21.15.31:0/4195985255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac819e520 con 0x7f8ac81006a0 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.838+0000 7f8abffff640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac8101890 0x7f8ac8196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:42.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.839+0000 7f8ac6ffd640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 0x7f8ac81969a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c970 tx=0x7f8ab800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.839+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8004220 con 0x7f8ac81006a0 2024-04-25T00:42:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.839+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ab80043c0 con 0x7f8ac81006a0 2024-04-25T00:42:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.839+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac819e810 con 0x7f8ac81006a0 2024-04-25T00:42:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.839+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800f450 con 0x7f8ac81006a0 2024-04-25T00:42:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.839+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac819edb0 con 0x7f8ac81006a0 2024-04-25T00:42:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.840+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac8107b30 con 0x7f8ac81006a0 2024-04-25T00:42:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.841+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8ab80079b0 con 0x7f8ac81006a0 2024-04-25T00:42:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.842+0000 7f8ac4ff9640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8ab40a2c10 0x7f8ab40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.842+0000 7f8abffff640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8ab40a2c10 0x7f8ab40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.842+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f8ab80ceec0 con 0x7f8ac81006a0 2024-04-25T00:42:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.842+0000 7f8abffff640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8ab40a2c10 0x7f8ab40a50d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8ab002a930 tx=0x7f8ab0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:42.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:42.846+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8ab8014030 con 0x7f8ac81006a0 2024-04-25T00:42:43.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.069+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8ac80623e0 con 0x7f8ac81006a0 2024-04-25T00:42:43.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.071+0000 7f8ac4ff9640 1 -- 172.21.15.31:0/4195985255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f8ab809bd90 con 0x7f8ac81006a0 2024-04-25T00:42:43.072 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:42:43.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8ab40a2c10 msgr2=0x7f8ab40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8ab40a2c10 0x7f8ab40a50d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8ab002a930 tx=0x7f8ab0005fd0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 msgr2=0x7f8ac81969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 0x7f8ac81969a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c970 tx=0x7f8ab800ce30 comp rx=0 tx=0).stop 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 shutdown_connections 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8102250 0x7f8ac819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac8101890 0x7f8ac8196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f8ab40a2c10 0x7f8ab40a50d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 --2- 172.21.15.31:0/4195985255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac81006a0 0x7f8ac81969a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 >> 172.21.15.31:0/4195985255 conn(0x7f8ac80fbc30 msgr2=0x7f8ac8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 shutdown_connections 2024-04-25T00:42:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.074+0000 7f8acd3ee640 1 -- 172.21.15.31:0/4195985255 wait complete. 2024-04-25T00:42:43.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:42:43.117 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:42 smithi169 ceph-mon[33112]: pgmap v725: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T00:42:43.118 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:42 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1843589364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:42:43.118 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:42 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2660425048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:42:43.119 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:42 smithi040 ceph-mon[32945]: pgmap v725: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T00:42:43.119 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:42 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1843589364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:42:43.120 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:42 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2660425048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 -- 172.21.15.31:0/512471276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 msgr2=0x7fab1c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 --2- 172.21.15.31:0/512471276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c100aa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fab18009900 tx=0x7fab1802a410 comp rx=0 tx=0).stop 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 -- 172.21.15.31:0/512471276 shutdown_connections 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 --2- 172.21.15.31:0/512471276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 0x7fab1c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 --2- 172.21.15.31:0/512471276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 0x7fab1c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 --2- 172.21.15.31:0/512471276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c100aa0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 -- 172.21.15.31:0/512471276 >> 172.21.15.31:0/512471276 conn(0x7fab1c0fbbf0 msgr2=0x7fab1c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 -- 172.21.15.31:0/512471276 shutdown_connections 2024-04-25T00:42:43.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.201+0000 7fab2364d640 1 -- 172.21.15.31:0/512471276 wait complete. 2024-04-25T00:42:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 Processor -- start 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 -- start start 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 0x7fab1c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 0x7fab1c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c10c180 con 0x7fab1c100680 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c10c000 con 0x7fab1c102230 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab2364d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c10c300 con 0x7fab1c101870 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49122/0 (socket says 172.21.15.31:49122) 2024-04-25T00:42:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 -- 172.21.15.31:0/1754992141 learned_addr learned my addr 172.21.15.31:0/1754992141 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab21bea640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 0x7fab1c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab20be8640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 0x7fab1c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 -- 172.21.15.31:0/1754992141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 msgr2=0x7fab1c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 0x7fab1c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 -- 172.21.15.31:0/1754992141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 msgr2=0x7fab1c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 0x7fab1c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab213e9640 1 -- 172.21.15.31:0/1754992141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c19f650 con 0x7fab1c100680 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab21bea640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 0x7fab1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.202+0000 7fab20be8640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 0x7fab1c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.203+0000 7fab213e9640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c197a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fab180314e0 tx=0x7fab18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.203+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1803f030 con 0x7fab1c100680 2024-04-25T00:42:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.203+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab18003aa0 con 0x7fab1c100680 2024-04-25T00:42:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.203+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18035070 con 0x7fab1c100680 2024-04-25T00:42:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.203+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c19f8e0 con 0x7fab1c100680 2024-04-25T00:42:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.203+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c19fd80 con 0x7fab1c100680 2024-04-25T00:42:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.204+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c19ffe0 con 0x7fab1c100680 2024-04-25T00:42:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.205+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab18042030 con 0x7fab1c100680 2024-04-25T00:42:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.206+0000 7fab127fc640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab080a2b90 0x7fab080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.206+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8707+0+0 (secure 0 0 0) 0x7fab180eff90 con 0x7fab1c100680 2024-04-25T00:42:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.206+0000 7fab20be8640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab080a2b90 0x7fab080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.207+0000 7fab20be8640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab080a2b90 0x7fab080a5050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab1c198f40 tx=0x7fab0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:43.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.209+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab180bcdd0 con 0x7fab1c100680 2024-04-25T00:42:43.372 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a[29163]: 2024-04-25T00:42:43.070+0000 7f5406b91700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a[68213]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a[68213]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mds[68237]: -- 172.21.15.31:0/553235878 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557339e99a40 con 0x557339e6cc00 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a[68213]: starting mds.a at 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j[66914]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j[66914]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mds[66920]: -- 172.21.15.31:0/602370178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56358de87a40 con 0x56358de5b000 2024-04-25T00:42:43.373 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j[66914]: starting mds.j at 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f[63748]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f[63748]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mds[63752]: -- 172.21.15.169:0/1329521056 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a23095a40 con 0x558a23068800 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f[63748]: starting mds.f at 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l[63436]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l[63436]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mds[63440]: -- 172.21.15.169:0/3836378802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b81467a40 con 0x556b8143a800 2024-04-25T00:42:43.390 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l[63436]: starting mds.l at 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mds[64379]: -- 172.21.15.169:0/3755154780 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56011d9e5a40 con 0x56011d9b8c00 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: starting mds.i at 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c[64064]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c[64064]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mds[64068]: -- 172.21.15.169:0/3931861992 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624ca541a40 con 0x5624ca514c00 2024-04-25T00:42:43.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c[64064]: starting mds.c at 2024-04-25T00:42:43.395 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k[62196]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.395 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k[62196]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.395 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k[62196]: starting mds.k at 2024-04-25T00:42:43.395 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b[63138]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.395 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b[63138]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mds[63142]: -- 172.21.15.40:0/2512967963 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648b3a2ba40 con 0x5648b39fec00 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b[63138]: starting mds.b at 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e[62516]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e[62516]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mds[62520]: -- 172.21.15.40:0/2938558409 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e5d073a40 con 0x562e5d047000 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e[62516]: starting mds.e at 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h[62827]: ignoring --setuser ceph since I am not root 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h[62827]: ignoring --setgroup ceph since I am not root 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mds[62831]: -- 172.21.15.40:0/1788054649 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e945f53a40 con 0x55e945f26c00 2024-04-25T00:42:43.396 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h[62827]: starting mds.h at 2024-04-25T00:42:43.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:43.435+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fab1c1a01b0 con 0x7fab1c100680 2024-04-25T00:42:44.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/2132365707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:44.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4195985255' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:42:44.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:42:44.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:42:44.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/4195985255' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:42:44.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:42:44.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T00:42:44.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:43 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1754992141' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:42:44.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.076+0000 7fab127fc640 1 -- 172.21.15.31:0/1754992141 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fab180bc6d0 con 0x7fab1c100680 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab080a2b90 msgr2=0x7fab080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab080a2b90 0x7fab080a5050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab1c198f40 tx=0x7fab0c005f90 comp rx=0 tx=0).stop 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 msgr2=0x7fab1c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c197a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fab180314e0 tx=0x7fab18004270 comp rx=0 tx=0).stop 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 shutdown_connections 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c102230 0x7fab1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab1c101870 0x7fab1c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7fab080a2b90 0x7fab080a5050 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 --2- 172.21.15.31:0/1754992141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab1c100680 0x7fab1c197a30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 >> 172.21.15.31:0/1754992141 conn(0x7fab1c0fbbf0 msgr2=0x7fab1c068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 shutdown_connections 2024-04-25T00:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.079+0000 7fab2364d640 1 -- 172.21.15.31:0/1754992141 wait complete. 2024-04-25T00:42:44.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:42:44.139 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/2132365707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4195985255' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/4195985255' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T00:42:44.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:43 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1754992141' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/2132365707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4195985255' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/4195985255' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T00:42:44.145 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:43 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1754992141' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:42:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 -- 172.21.15.31:0/563021768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4104bf0 msgr2=0x7f85b4106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/563021768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4104bf0 0x7f85b4106fe0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009900 tx=0x7f85b002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 -- 172.21.15.31:0/563021768 shutdown_connections 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/563021768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b4107520 0x7f85b4109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/563021768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4104bf0 0x7f85b4106fe0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/563021768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b41022c0 0x7f85b41046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 -- 172.21.15.31:0/563021768 >> 172.21.15.31:0/563021768 conn(0x7f85b40fbbf0 msgr2=0x7f85b40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 -- 172.21.15.31:0/563021768 shutdown_connections 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 -- 172.21.15.31:0/563021768 wait complete. 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 Processor -- start 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 -- start start 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.201+0000 7f85bb9ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b41022c0 0x7f85b4101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85bb9ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b4104bf0 0x7f85b4100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85bb9ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 0x7f85b41007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85bb9ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85b410d310 con 0x7f85b4107520 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85bb9ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85b410d190 con 0x7f85b4104bf0 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85bb9ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85b410d490 con 0x7f85b41022c0 2024-04-25T00:42:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 0x7f85b41007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 0x7f85b41007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49138/0 (socket says 172.21.15.31:49138) 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 -- 172.21.15.31:0/3986597293 learned_addr learned my addr 172.21.15.31:0/3986597293 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b978a640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b41022c0 0x7f85b4101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b8f89640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b4104bf0 0x7f85b4100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 -- 172.21.15.31:0/3986597293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b41022c0 msgr2=0x7f85b4101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b41022c0 0x7f85b4101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 -- 172.21.15.31:0/3986597293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b4104bf0 msgr2=0x7f85b4100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b4104bf0 0x7f85b4100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 -- 172.21.15.31:0/3986597293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4101040 con 0x7f85b4107520 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85b9f8b640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 0x7f85b41007b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b970 tx=0x7f85ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac019070 con 0x7f85b4107520 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ac014440 con 0x7f85b4107520 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b41a1b60 con 0x7f85b4107520 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.202+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0075d0 con 0x7f85b4107520 2024-04-25T00:42:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.203+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b41a2030 con 0x7f85b4107520 2024-04-25T00:42:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.203+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b4068560 con 0x7f85b4107520 2024-04-25T00:42:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.205+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85ac01c070 con 0x7f85b4107520 2024-04-25T00:42:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.206+0000 7f85a67fc640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85a00a2b90 0x7f85a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.206+0000 7f85b978a640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85a00a2b90 0x7f85a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.206+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8707+0+0 (secure 0 0 0) 0x7f85ac0cecd0 con 0x7f85b4107520 2024-04-25T00:42:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.207+0000 7f85b978a640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85a00a2b90 0x7f85a00a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f85a800abe0 tx=0x7f85a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.209+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f85ac09ba50 con 0x7f85b4107520 2024-04-25T00:42:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:44.396+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85b40fd6e0 con 0x7f85b4107520 2024-04-25T00:42:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: pgmap v727: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T00:42:44.895 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: Cluster is now healthy 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/1754992141' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6834/4068917989,v1:172.21.15.31:6840/4068917989] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.31:6835/3246914239,v1:172.21.15.31:6841/3246914239] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6833/887610302,v1:172.21.15.169:6835/887610302] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6832/2458615074,v1:172.21.15.169:6834/2458615074] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6832/676352863,v1:172.21.15.40:6834/676352863] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6833/1248756454,v1:172.21.15.40:6835/1248756454] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6836/3376493417,v1:172.21.15.169:6838/3376493417] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.169:6837/433534006,v1:172.21.15.169:6839/433534006] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6837/611055186,v1:172.21.15.40:6839/611055186] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: mds.? [v2:172.21.15.40:6836/235601912,v1:172.21.15.40:6838/235601912] up:boot 2024-04-25T00:42:44.896 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: fsmap 12 up:standby 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:42:44.897 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:44 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3986597293' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: pgmap v727: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: Cluster is now healthy 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/1754992141' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6834/4068917989,v1:172.21.15.31:6840/4068917989] up:boot 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.31:6835/3246914239,v1:172.21.15.31:6841/3246914239] up:boot 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6833/887610302,v1:172.21.15.169:6835/887610302] up:boot 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6832/2458615074,v1:172.21.15.169:6834/2458615074] up:boot 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6832/676352863,v1:172.21.15.40:6834/676352863] up:boot 2024-04-25T00:42:45.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6833/1248756454,v1:172.21.15.40:6835/1248756454] up:boot 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6836/3376493417,v1:172.21.15.169:6838/3376493417] up:boot 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.169:6837/433534006,v1:172.21.15.169:6839/433534006] up:boot 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6837/611055186,v1:172.21.15.40:6839/611055186] up:boot 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: mds.? [v2:172.21.15.40:6836/235601912,v1:172.21.15.40:6838/235601912] up:boot 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: fsmap 12 up:standby 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:42:45.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:44 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3986597293' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:42:45.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.078+0000 7f85a67fc640 1 -- 172.21.15.31:0/3986597293 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f85ac09b350 con 0x7f85b4107520 2024-04-25T00:42:45.079 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:42:45.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.081+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85a00a2b90 msgr2=0x7f85a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.081+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85a00a2b90 0x7f85a00a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f85a800abe0 tx=0x7f85a800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.081+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 msgr2=0x7f85b41007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.081+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 0x7f85b41007b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b970 tx=0x7f85ac00bd70 comp rx=0 tx=0).stop 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 shutdown_connections 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f85a00a2b90 0x7f85a00a5050 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f85a800abe0 tx=0x7f85a800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b4107520 0x7f85b41007b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b4104bf0 0x7f85b4100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 --2- 172.21.15.31:0/3986597293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b41022c0 0x7f85b4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 >> 172.21.15.31:0/3986597293 conn(0x7f85b40fbbf0 msgr2=0x7f85b4102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 shutdown_connections 2024-04-25T00:42:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.082+0000 7f85bb9ee640 1 -- 172.21.15.31:0/3986597293 wait complete. 2024-04-25T00:42:45.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: pgmap v727: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: Cluster is now healthy 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/1754992141' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6834/4068917989,v1:172.21.15.31:6840/4068917989] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.31:6835/3246914239,v1:172.21.15.31:6841/3246914239] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6833/887610302,v1:172.21.15.169:6835/887610302] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6832/2458615074,v1:172.21.15.169:6834/2458615074] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6832/676352863,v1:172.21.15.40:6834/676352863] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6833/1248756454,v1:172.21.15.40:6835/1248756454] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6836/3376493417,v1:172.21.15.169:6838/3376493417] up:boot 2024-04-25T00:42:45.140 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.169:6837/433534006,v1:172.21.15.169:6839/433534006] up:boot 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6837/611055186,v1:172.21.15.40:6839/611055186] up:boot 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: mds.? [v2:172.21.15.40:6836/235601912,v1:172.21.15.40:6838/235601912] up:boot 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: fsmap 12 up:standby 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='mgr.14150 172.21.15.31:0/2039828930' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:42:45.141 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:44 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3986597293' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/1011011265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 msgr2=0x7f9f54100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/1011011265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f54100a90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009900 tx=0x7f9f5002a410 comp rx=0 tx=0).stop 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/1011011265 shutdown_connections 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/1011011265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 0x7f9f54108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/1011011265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f54101860 0x7f9f54101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/1011011265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f54100a90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/1011011265 >> 172.21.15.31:0/1011011265 conn(0x7f9f540fbbe0 msgr2=0x7f9f540fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/1011011265 shutdown_connections 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.214+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/1011011265 wait complete. 2024-04-25T00:42:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 Processor -- start 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 -- start start 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f5406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f54101860 0x7f9f5419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 0x7f9f5419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5410c1a0 con 0x7f9f54100670 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5410c020 con 0x7f9f54102220 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5b8e1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5410c320 con 0x7f9f54101860 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5967d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f5406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5967d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f5406a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49144/0 (socket says 172.21.15.31:49144) 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f5967d640 1 -- 172.21.15.31:0/3935661551 learned_addr learned my addr 172.21.15.31:0/3935661551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.215+0000 7f9f59e7e640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 0x7f9f5419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f58e7c640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f54101860 0x7f9f5419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5967d640 1 -- 172.21.15.31:0/3935661551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f54101860 msgr2=0x7f9f5419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5967d640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f54101860 0x7f9f5419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5967d640 1 -- 172.21.15.31:0/3935661551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 msgr2=0x7f9f5419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5967d640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 0x7f9f5419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5967d640 1 -- 172.21.15.31:0/3935661551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5419f620 con 0x7f9f54100670 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f59e7e640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 0x7f9f5419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5967d640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f5406a910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9f500314e0 tx=0x7f9f500042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5003f030 con 0x7f9f54100670 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5419f8b0 con 0x7f9f54100670 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f50003cb0 con 0x7f9f54100670 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50035070 con 0x7f9f54100670 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.216+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5419fdf0 con 0x7f9f54100670 2024-04-25T00:42:45.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.217+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f54107a80 con 0x7f9f54100670 2024-04-25T00:42:45.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.218+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f50042030 con 0x7f9f54100670 2024-04-25T00:42:45.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.219+0000 7f9f4a7fc640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9f400a2b90 0x7f9f400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:45.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.219+0000 7f9f58e7c640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9f400a2b90 0x7f9f400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:45.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.219+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8145+0+0 (secure 0 0 0) 0x7f9f500ef730 con 0x7f9f54100670 2024-04-25T00:42:45.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.219+0000 7f9f58e7c640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9f400a2b90 0x7f9f400a5050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005fd0 tx=0x7f9f44005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:45.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.223+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f500bc690 con 0x7f9f54100670 2024-04-25T00:42:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:45.409+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9f54062340 con 0x7f9f54100670 2024-04-25T00:42:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.084+0000 7f9f4a7fc640 1 -- 172.21.15.31:0/3935661551 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f9f500bbf90 con 0x7f9f54100670 2024-04-25T00:42:46.085 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.089+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9f400a2b90 msgr2=0x7f9f400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.089+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9f400a2b90 0x7f9f400a5050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005fd0 tx=0x7f9f44005850 comp rx=0 tx=0).stop 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.089+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 msgr2=0x7f9f5406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.089+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f5406a910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9f500314e0 tx=0x7f9f500042e0 comp rx=0 tx=0).stop 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 shutdown_connections 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f54102220 0x7f9f5419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f54101860 0x7f9f5419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:6800/2153348461,v1:172.21.15.31:6801/2153348461] conn(0x7f9f400a2b90 0x7f9f400a5050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 --2- 172.21.15.31:0/3935661551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f54100670 0x7f9f5406a910 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 >> 172.21.15.31:0/3935661551 conn(0x7f9f540fbbe0 msgr2=0x7f9f54104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 shutdown_connections 2024-04-25T00:42:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T00:42:46.090+0000 7f9f5b8e1640 1 -- 172.21.15.31:0/3935661551 wait complete. 2024-04-25T00:42:46.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:42:46.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:42:46.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:42:46.203 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T00:42:46.203 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:42:46.220 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:42:46.237 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:42:46.260 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T00:42:46.260 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T00:42:46.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:42:46.389 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:46 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3986597293' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:42:46.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:46 smithi169 ceph-mon[33112]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:42:46.390 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:46 smithi169 ceph-mon[33112]: from='client.? 172.21.15.31:0/3935661551' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:42:46.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:46 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3986597293' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:42:46.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:46 smithi040 ceph-mon[32945]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:42:46.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:46 smithi040 ceph-mon[32945]: from='client.? 172.21.15.31:0/3935661551' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:42:46.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3986597293' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:42:46.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 ceph-mon[29167]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:42:46.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 ceph-mon[29167]: from='client.? 172.21.15.31:0/3935661551' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:42:46.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 systemd[1]: Stopping Ceph mon.a for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a[29163]: 2024-04-25T00:42:46.580+0000 7f540c39c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:42:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a[29163]: 2024-04-25T00:42:46.580+0000 7f540c39c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T00:42:47.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:46 smithi031 podman[85888]: 2024-04-25 00:42:46.794828657 +0000 UTC m=+0.337754528 container died 5218831c95b631d2c27e7858d4ccba3740a89f9da9cb58c5ceec0ef4edc18f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:42:47.474 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:47 smithi031 podman[85888]: 2024-04-25 00:42:47.164510577 +0000 UTC m=+0.707436440 container cleanup 5218831c95b631d2c27e7858d4ccba3740a89f9da9cb58c5ceec0ef4edc18f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:42:47.474 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 00:42:47 smithi031 bash[85888]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:42:47.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T00:42:47.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:42:47.693 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T00:42:47.694 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T00:42:47.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.b 2024-04-25T00:42:48.012 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:47 smithi040 systemd[1]: Stopping Ceph mon.b for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b[32941]: 2024-04-25T00:42:48.010+0000 7f59e4d4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:42:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b[32941]: 2024-04-25T00:42:48.010+0000 7f59e4d4e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:42:48.395 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:48 smithi040 podman[67678]: 2024-04-25 00:42:48.196086502 +0000 UTC m=+0.290738234 container died b28e66123773c25c33fff9905cb17391123d4860c0cd80972342982414b63d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-25T00:42:48.813 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:48 smithi040 podman[67678]: 2024-04-25 00:42:48.58742228 +0000 UTC m=+0.682073955 container cleanup b28e66123773c25c33fff9905cb17391123d4860c0cd80972342982414b63d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T00:42:48.814 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 00:42:48 smithi040 bash[67678]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-b 2024-04-25T00:42:48.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T00:42:49.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:42:49.009 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T00:42:49.009 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T00:42:49.009 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.c 2024-04-25T00:42:49.317 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:49 smithi169 systemd[1]: Stopping Ceph mon.c for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:49.639 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-c[33095]: 2024-04-25T00:42:49.316+0000 7f043ff24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:42:49.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-c[33095]: 2024-04-25T00:42:49.316+0000 7f043ff24700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:42:49.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:49 smithi169 podman[69506]: 2024-04-25 00:42:49.562009017 +0000 UTC m=+0.357715190 container died 3fe2bfd0089e3e6459916ba8e5e5975a09e87d5cffaba28f10a9ddd248cd0211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-c, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True) 2024-04-25T00:42:50.210 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:49 smithi169 podman[69506]: 2024-04-25 00:42:49.992940898 +0000 UTC m=+0.788647068 container cleanup 3fe2bfd0089e3e6459916ba8e5e5975a09e87d5cffaba28f10a9ddd248cd0211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T00:42:50.211 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 00:42:49 smithi169 bash[69506]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mon-c 2024-04-25T00:42:50.430 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T00:42:50.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:42:50.474 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T00:42:50.475 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T00:42:50.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:42:50.772 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:42:50 smithi031 systemd[1]: Stopping Ceph mgr.x for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:51.104 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:42:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x[29399]: 2024-04-25T00:42:50.785+0000 7feaaa151700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:42:51.104 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:42:50 smithi031 podman[86011]: 2024-04-25 00:42:50.817000324 +0000 UTC m=+0.145608607 container died 9b1f6788656b831ae01c20f6b5de954f634068a950e31a5d7349dc7bce1605e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:42:51.438 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:42:51 smithi031 podman[86011]: 2024-04-25 00:42:51.203756828 +0000 UTC m=+0.532365109 container cleanup 9b1f6788656b831ae01c20f6b5de954f634068a950e31a5d7349dc7bce1605e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:42:51.438 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 00:42:51 smithi031 bash[86011]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-x 2024-04-25T00:42:51.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T00:42:51.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:42:51.615 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T00:42:51.615 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T00:42:51.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.y 2024-04-25T00:42:51.895 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:42:51 smithi040 systemd[1]: Stopping Ceph mgr.y for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:52.244 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:42:51 smithi040 podman[67801]: 2024-04-25 00:42:51.955199631 +0000 UTC m=+0.140499615 container died 889156ed9c113922f9b0960f181a4791dbbc55b5ab4b1f0b8f29dc303b4b013b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T00:42:52.594 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:42:52 smithi040 podman[67801]: 2024-04-25 00:42:52.318220099 +0000 UTC m=+0.503520081 container cleanup 889156ed9c113922f9b0960f181a4791dbbc55b5ab4b1f0b8f29dc303b4b013b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:42:52.594 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 00:42:52 smithi040 bash[67801]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-y 2024-04-25T00:42:52.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T00:42:52.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:42:52.781 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T00:42:52.781 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T00:42:52.781 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.z 2024-04-25T00:42:53.127 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:42:52 smithi169 systemd[1]: Stopping Ceph mgr.z for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:53.389 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:42:53 smithi169 podman[69633]: 2024-04-25 00:42:53.12770774 +0000 UTC m=+0.155844308 container died cfba88e51db82ae8ae5c5bf9cdc67e8327267d9d12bdcbefb72ebaa3b1d82a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:42:53.785 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:42:53 smithi169 podman[69633]: 2024-04-25 00:42:53.533350491 +0000 UTC m=+0.561487070 container cleanup cfba88e51db82ae8ae5c5bf9cdc67e8327267d9d12bdcbefb72ebaa3b1d82a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:42:53.785 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 00:42:53 smithi169 bash[69633]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mgr-z 2024-04-25T00:42:53.885 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T00:42:53.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:42:53.919 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T00:42:53.919 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T00:42:53.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.0 2024-04-25T00:42:54.234 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:42:53 smithi031 systemd[1]: Stopping Ceph osd.0 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:42:54.522 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:42:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0[41409]: 2024-04-25T00:42:54.233+0000 7fe477968700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:42:54.522 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:42:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0[41409]: 2024-04-25T00:42:54.233+0000 7fe477968700 -1 osd.0 94 *** Got signal Terminated *** 2024-04-25T00:42:54.522 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:42:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0[41409]: 2024-04-25T00:42:54.233+0000 7fe477968700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:00.023 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:42:59 smithi031 podman[86136]: 2024-04-25 00:42:59.706339175 +0000 UTC m=+5.590878760 container died d640216608efd8b028e2ddfd56cba16d8681b675a71a37a476a5fe97eed58182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T00:43:00.351 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:00 smithi031 podman[86136]: 2024-04-25 00:43:00.124730582 +0000 UTC m=+6.009270180 container cleanup d640216608efd8b028e2ddfd56cba16d8681b675a71a37a476a5fe97eed58182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:43:00.351 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:00 smithi031 bash[86136]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0 2024-04-25T00:43:00.772 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:00 smithi031 podman[86154]: 2024-04-25 00:43:00.351283709 +0000 UTC m=+0.641579111 container remove d640216608efd8b028e2ddfd56cba16d8681b675a71a37a476a5fe97eed58182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:01.082 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:00 smithi031 podman[86246]: 2024-04-25 00:43:00.712173081 +0000 UTC m=+0.018627148 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:01.083 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:00 smithi031 podman[86246]: 2024-04-25 00:43:00.829546244 +0000 UTC m=+0.136000334 container create 672e1c7467e928029b0ab97dec118c1fd023984dee4b9bcadf21ca910c018097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:43:01.388 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:01 smithi031 podman[86246]: 2024-04-25 00:43:01.082910143 +0000 UTC m=+0.389364204 container init 672e1c7467e928029b0ab97dec118c1fd023984dee4b9bcadf21ca910c018097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:01.388 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:01 smithi031 podman[86246]: 2024-04-25 00:43:01.08652075 +0000 UTC m=+0.392974798 container start 672e1c7467e928029b0ab97dec118c1fd023984dee4b9bcadf21ca910c018097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T00:43:01.388 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:01 smithi031 podman[86246]: 2024-04-25 00:43:01.182539341 +0000 UTC m=+0.488993417 container attach 672e1c7467e928029b0ab97dec118c1fd023984dee4b9bcadf21ca910c018097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:43:01.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 00:43:01 smithi031 podman[86246]: 2024-04-25 00:43:01.38823876 +0000 UTC m=+0.694692812 container died 672e1c7467e928029b0ab97dec118c1fd023984dee4b9bcadf21ca910c018097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:43:01.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T00:43:02.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:02.044 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T00:43:02.044 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T00:43:02.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.1 2024-04-25T00:43:02.365 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:02 smithi040 systemd[1]: Stopping Ceph osd.1 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:02.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1[37546]: 2024-04-25T00:43:02.364+0000 7f47118bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:02.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1[37546]: 2024-04-25T00:43:02.364+0000 7f47118bf700 -1 osd.1 94 *** Got signal Terminated *** 2024-04-25T00:43:02.645 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1[37546]: 2024-04-25T00:43:02.364+0000 7f47118bf700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:08.141 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:07 smithi040 podman[67927]: 2024-04-25 00:43:07.858058331 +0000 UTC m=+5.608723159 container died 1d1b75f5d807980cdf2eb47c32b494633c0f6cf82631467d23cfdf8fd6bd8467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-25T00:43:08.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:08 smithi040 podman[67927]: 2024-04-25 00:43:08.231094481 +0000 UTC m=+5.981759314 container cleanup 1d1b75f5d807980cdf2eb47c32b494633c0f6cf82631467d23cfdf8fd6bd8467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:43:08.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:08 smithi040 bash[67927]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1 2024-04-25T00:43:08.861 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:08 smithi040 podman[67956]: 2024-04-25 00:43:08.524898999 +0000 UTC m=+0.668045906 container remove 1d1b75f5d807980cdf2eb47c32b494633c0f6cf82631467d23cfdf8fd6bd8467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:43:09.112 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:08 smithi040 podman[68046]: 2024-04-25 00:43:08.76434005 +0000 UTC m=+0.018524100 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:09.112 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:08 smithi040 podman[68046]: 2024-04-25 00:43:08.867655038 +0000 UTC m=+0.121839113 container create 7b7999267b4becdac490e2beee195f8ca1f892c412bf707e2def8c818bdb2e0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:43:09.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:09 smithi040 podman[68046]: 2024-04-25 00:43:09.112525205 +0000 UTC m=+0.366709240 container init 7b7999267b4becdac490e2beee195f8ca1f892c412bf707e2def8c818bdb2e0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-25T00:43:09.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:09 smithi040 podman[68046]: 2024-04-25 00:43:09.116330276 +0000 UTC m=+0.370514307 container start 7b7999267b4becdac490e2beee195f8ca1f892c412bf707e2def8c818bdb2e0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T00:43:09.395 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:09 smithi040 podman[68046]: 2024-04-25 00:43:09.220785251 +0000 UTC m=+0.474969287 container attach 7b7999267b4becdac490e2beee195f8ca1f892c412bf707e2def8c818bdb2e0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T00:43:09.716 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 00:43:09 smithi040 podman[68046]: 2024-04-25 00:43:09.418313285 +0000 UTC m=+0.672497315 container died 7b7999267b4becdac490e2beee195f8ca1f892c412bf707e2def8c818bdb2e0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:43:10.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T00:43:10.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:10.058 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T00:43:10.058 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T00:43:10.059 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.2 2024-04-25T00:43:10.380 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:10 smithi169 systemd[1]: Stopping Ceph osd.2 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:10.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2[38382]: 2024-04-25T00:43:10.379+0000 7fd7d2510700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:10.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2[38382]: 2024-04-25T00:43:10.379+0000 7fd7d2510700 -1 osd.2 94 *** Got signal Terminated *** 2024-04-25T00:43:10.640 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2[38382]: 2024-04-25T00:43:10.379+0000 7fd7d2510700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:16.081 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:15 smithi169 podman[69770]: 2024-04-25 00:43:15.772604365 +0000 UTC m=+5.516552169 container died 9f5cd8848c247845af51a620194475fa84879836f53c79723506d1cc7f807030 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:16.389 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:16 smithi169 podman[69770]: 2024-04-25 00:43:16.180928034 +0000 UTC m=+5.924875841 container cleanup 9f5cd8848c247845af51a620194475fa84879836f53c79723506d1cc7f807030 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-25T00:43:16.390 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:16 smithi169 bash[69770]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2 2024-04-25T00:43:16.757 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:16 smithi169 podman[69787]: 2024-04-25 00:43:16.440708641 +0000 UTC m=+0.658998240 container remove 9f5cd8848c247845af51a620194475fa84879836f53c79723506d1cc7f807030 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T00:43:17.053 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:16 smithi169 podman[69876]: 2024-04-25 00:43:16.659845825 +0000 UTC m=+0.018158893 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:17.053 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:16 smithi169 podman[69876]: 2024-04-25 00:43:16.775061089 +0000 UTC m=+0.133374159 container create a15829cd053a76874e72c5474fc39f48b4dc745467db09ca908be04a26c981c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T00:43:17.053 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:17 smithi169 podman[69876]: 2024-04-25 00:43:17.053246878 +0000 UTC m=+0.411559956 container init a15829cd053a76874e72c5474fc39f48b4dc745467db09ca908be04a26c981c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:43:17.346 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:17 smithi169 podman[69876]: 2024-04-25 00:43:17.0570403 +0000 UTC m=+0.415353377 container start a15829cd053a76874e72c5474fc39f48b4dc745467db09ca908be04a26c981c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T00:43:17.346 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:17 smithi169 podman[69876]: 2024-04-25 00:43:17.144908393 +0000 UTC m=+0.503221464 container attach a15829cd053a76874e72c5474fc39f48b4dc745467db09ca908be04a26c981c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:43:17.639 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 00:43:17 smithi169 podman[69876]: 2024-04-25 00:43:17.346689829 +0000 UTC m=+0.705002901 container died a15829cd053a76874e72c5474fc39f48b4dc745467db09ca908be04a26c981c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T00:43:17.981 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T00:43:18.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:18.020 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T00:43:18.020 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T00:43:18.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.3 2024-04-25T00:43:18.339 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:18 smithi031 systemd[1]: Stopping Ceph osd.3 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:18.772 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:18 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3[48377]: 2024-04-25T00:43:18.338+0000 7f7ba2127700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:18.772 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:18 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3[48377]: 2024-04-25T00:43:18.338+0000 7f7ba2127700 -1 osd.3 94 *** Got signal Terminated *** 2024-04-25T00:43:18.772 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:18 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3[48377]: 2024-04-25T00:43:18.338+0000 7f7ba2127700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:21.522 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:21 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:21.029+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:22.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:22.001+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:22.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:21.960+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:22.978 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3[48377]: 2024-04-25T00:43:22.697+0000 7f7b9a713700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.803767+0000 front 2024-04-25T00:42:58.804252+0000 (oldest deadline 2024-04-25T00:43:22.303943+0000) 2024-04-25T00:43:23.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:22 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:22.978+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:23.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:22.991+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:24.022 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:23 smithi031 podman[86352]: 2024-04-25 00:43:23.751253028 +0000 UTC m=+5.532717929 container died 09b72be50d9d44306b58dac0f2e23c801b06e8023f24258a8549cb704cd896d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T00:43:24.022 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:23 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:23.952+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:24.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:24.021+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:24.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:23.989+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:24.448 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:24 smithi031 podman[86352]: 2024-04-25 00:43:24.122185402 +0000 UTC m=+5.903650287 container cleanup 09b72be50d9d44306b58dac0f2e23c801b06e8023f24258a8549cb704cd896d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-25T00:43:24.448 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:24 smithi031 bash[86352]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3 2024-04-25T00:43:24.772 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:24 smithi031 podman[86381]: 2024-04-25 00:43:24.448545088 +0000 UTC m=+0.688923433 container remove 09b72be50d9d44306b58dac0f2e23c801b06e8023f24258a8549cb704cd896d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T00:43:24.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:24 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:24.497+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:25.094 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:24 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:24.952+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:25.095 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:24 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:24.913+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:25.095 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:24 smithi031 podman[86473]: 2024-04-25 00:43:24.712878966 +0000 UTC m=+0.018493866 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:25.095 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:24 smithi031 podman[86473]: 2024-04-25 00:43:24.824788412 +0000 UTC m=+0.130403341 container create 62adaba0f04c8ef222cd51d865462f980e9fdd3298b1be638b9ff5e8ad4f8dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:25.389 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:25.020+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:25.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:25.022+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:25.407 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:25 smithi031 podman[86473]: 2024-04-25 00:43:25.094545151 +0000 UTC m=+0.400160045 container init 62adaba0f04c8ef222cd51d865462f980e9fdd3298b1be638b9ff5e8ad4f8dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:43:25.407 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:25 smithi031 podman[86473]: 2024-04-25 00:43:25.098644769 +0000 UTC m=+0.404259663 container start 62adaba0f04c8ef222cd51d865462f980e9fdd3298b1be638b9ff5e8ad4f8dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T00:43:25.407 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:25 smithi031 podman[86473]: 2024-04-25 00:43:25.202755536 +0000 UTC m=+0.508370442 container attach 62adaba0f04c8ef222cd51d865462f980e9fdd3298b1be638b9ff5e8ad4f8dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:43:25.697 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 00:43:25 smithi031 podman[86473]: 2024-04-25 00:43:25.407764107 +0000 UTC m=+0.713379007 container died 62adaba0f04c8ef222cd51d865462f980e9fdd3298b1be638b9ff5e8ad4f8dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:43:25.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:25 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:25.504+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:25.987 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:25.913+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:26.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T00:43:26.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:26.091 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T00:43:26.092 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T00:43:26.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.4 2024-04-25T00:43:26.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:25 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:25.987+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:26.389 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:26.005+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:26.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:25.991+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:26.394 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:26 smithi040 systemd[1]: Stopping Ceph osd.4 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:26.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:26 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:26.463+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:26.895 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:26 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4[44210]: 2024-04-25T00:43:26.415+0000 7fb3becc3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:26.895 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:26 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4[44210]: 2024-04-25T00:43:26.415+0000 7fb3becc3700 -1 osd.4 94 *** Got signal Terminated *** 2024-04-25T00:43:26.895 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:26 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4[44210]: 2024-04-25T00:43:26.415+0000 7fb3becc3700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:27.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:26 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:26.978+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:27.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:26 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:26.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:27.389 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:26.997+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:27.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:27.016+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:27.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:27 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:27.447+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:28.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:27 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:27.992+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:28.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:27 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:27.988+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:28.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:28.003+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:28.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:28.013+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:28.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:28 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:28.415+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:29.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:28.998+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:29.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:28 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:28.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:29.389 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:28.975+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:29.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:28 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:28.979+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:29.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:29 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:29.443+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:29.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:29 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:29.443+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:30.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:29.978+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:30.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:29 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:29.916+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:30.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:29 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:29.953+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:30.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:29 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:29.939+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:30.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:29 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:29.939+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:30.714 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:30.452+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:30.714 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:30.452+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:31.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:30 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:30.713+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:31.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:30 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:30.925+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:31.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:30 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:30.929+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:31.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:30 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:30.929+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:31.291 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:30.957+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:31.291 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:30.911+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:31.291 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:30 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:30.911+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:31.755 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:31.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:31.755 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:31.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:32.146 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:31 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:31.754+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:32.146 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:31 smithi040 podman[68153]: 2024-04-25 00:43:31.951209869 +0000 UTC m=+5.661980323 container died ff85eae39914d9a8b7b8eb4c0f90550ff54caccb4358765621230d484077d590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T00:43:32.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:31 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:31.934+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:32.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:31 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:31.958+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:32.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:31 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:31.958+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:32.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:31 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:31.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:32.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:32 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:32.037+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:32.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:31 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:31.912+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:32.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:31 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:31.912+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:32.429 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:32 smithi040 podman[68153]: 2024-04-25 00:43:32.327752769 +0000 UTC m=+6.038523221 container cleanup ff85eae39914d9a8b7b8eb4c0f90550ff54caccb4358765621230d484077d590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:32.429 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:32 smithi040 bash[68153]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4 2024-04-25T00:43:32.784 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:32 smithi040 podman[68171]: 2024-04-25 00:43:32.638258168 +0000 UTC m=+0.682362514 container remove ff85eae39914d9a8b7b8eb4c0f90550ff54caccb4358765621230d484077d590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:43:32.784 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:32.428+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:32.784 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:32.428+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:33.089 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:32 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:32.782+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:33.089 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:32 smithi040 podman[68260]: 2024-04-25 00:43:32.861575985 +0000 UTC m=+0.018737426 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:33.089 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:33 smithi040 podman[68260]: 2024-04-25 00:43:33.089206032 +0000 UTC m=+0.246367465 container create ca57a2e2bee3b7a91a768e496da306a6fed27863b6b2d016e80e369d2bee9c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T00:43:33.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:32 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:32.988+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:33.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:32 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:32.988+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:33.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:32 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:32.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:33.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:32 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:32.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:33.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:33 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:33.001+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:33.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:32 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:32.976+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:33.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:32 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:32.908+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:33.390 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:32 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:32.908+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:33.645 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:33 smithi040 podman[68260]: 2024-04-25 00:43:33.358436598 +0000 UTC m=+0.515598028 container init ca57a2e2bee3b7a91a768e496da306a6fed27863b6b2d016e80e369d2bee9c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T00:43:33.646 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:33 smithi040 podman[68260]: 2024-04-25 00:43:33.361625692 +0000 UTC m=+0.518787130 container start ca57a2e2bee3b7a91a768e496da306a6fed27863b6b2d016e80e369d2bee9c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-25T00:43:33.646 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:33 smithi040 podman[68260]: 2024-04-25 00:43:33.466822816 +0000 UTC m=+0.623984259 container attach ca57a2e2bee3b7a91a768e496da306a6fed27863b6b2d016e80e369d2bee9c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:43:33.646 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:33.380+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:33.646 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:33.380+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:33.953 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:33 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:33.747+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:33.953 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 00:43:33 smithi040 podman[68260]: 2024-04-25 00:43:33.666746323 +0000 UTC m=+0.823907761 container died ca57a2e2bee3b7a91a768e496da306a6fed27863b6b2d016e80e369d2bee9c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:43:34.139 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:34.013+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:34.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:34.031+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:34.140 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:33 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:33.869+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:34.140 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:33 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:33.869+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:34.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T00:43:34.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:34 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:34.026+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:34.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:34 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:34.026+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:34.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:33 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:33.914+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:34.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:33 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:33.914+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:34.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:34.300 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T00:43:34.300 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T00:43:34.300 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.5 2024-04-25T00:43:34.619 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:34 smithi169 systemd[1]: Stopping Ceph osd.5 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:34.619 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:34.618+0000 7fc622ca0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:34.619 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:34.618+0000 7fc622ca0700 -1 osd.5 94 *** Got signal Terminated *** 2024-04-25T00:43:34.619 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:34.618+0000 7fc622ca0700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:34.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:34 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:34.378+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:34.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:34 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:34.378+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:34.889 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:34.833+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:34.889 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:34 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:34.833+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:35.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:34 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:34.738+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:35.181 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:35 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:35.060+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:35.181 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:35 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:35.060+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:35.182 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:34 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:34.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:35.182 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:34 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:34.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:35.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:35 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:35.058+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:35.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:35 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:35.041+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:35.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:35 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:35.359+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:35.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:35 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:35.359+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:36.083 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:35 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:35.816+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:36.083 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:35 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:35.816+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:36.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:35 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:35.759+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:36.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:36 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:36.069+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:36.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:36 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:36.069+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:36.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:35 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:35.949+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:36.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:35 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:35.949+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:36.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:36 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:36.089+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:36.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:36 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:36.082+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:36.720 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:36 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:36.400+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:36.720 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:36 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:36.400+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:36.720 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:36 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:36.400+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:37.040 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:36 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:36.770+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:37.040 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:36 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:36.770+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:37.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:36 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:36.719+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:37.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:37 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:37.064+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:37.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:37 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:37.064+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:37.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:36 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:36.928+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:37.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:36 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:36.928+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:37.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:37.121+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:37.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:37.040+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:37.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:37.040+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:37.765 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:37 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:37.417+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:37.765 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:37 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:37.417+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:37.765 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:37 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:37.417+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:37.996 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:37.732+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:37.997 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:37.732+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:38.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:37 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:37.764+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:38.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:38.061+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:38.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:38.061+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:38.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:37 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:37.936+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:38.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:37 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:37.936+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:38.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:38 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:38.099+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:38.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:38 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:38.099+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:38.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:37.996+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:38.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:37 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:37.996+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:38.740 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:38 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:38.409+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:38.740 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:38 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:38.409+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:38.740 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:38 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:38.409+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:39.135 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:38 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:38.764+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.956936+0000 front 2024-04-25T00:42:58.956942+0000 (oldest deadline 2024-04-25T00:43:23.056709+0000) 2024-04-25T00:43:39.135 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:38 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-25T00:43:38.764+0000 7fc61b28c700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.458006+0000 front 2024-04-25T00:43:06.458043+0000 (oldest deadline 2024-04-25T00:43:29.358139+0000) 2024-04-25T00:43:39.135 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:38 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:38.965+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:39.135 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:38 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:38.965+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:39.144 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:38 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:38.739+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:39.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:38 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:38.739+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:39.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:39 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:39.019+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:39.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:39 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:39.019+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:39.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:38.971+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:39.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:38 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:38.971+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:39.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:39 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:39.134+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:39.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:39 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:39.134+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:39.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:39 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:39.383+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:39.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:39 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:39.383+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:39.645 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:39 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:39.383+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:40.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:39 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:39.745+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:40.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:39 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:39.745+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:40.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:40.013+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:40.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:40.013+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:40.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:40.013+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:40.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:39.999+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:40.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:39.999+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:40.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:39.999+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:40.294 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:40 smithi169 podman[69980]: 2024-04-25 00:43:40.000252694 +0000 UTC m=+5.502940807 container died e5491819dbec2c8423a3fdfcfc62435e7e0bced2bb1891203d0786a04a39128a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:43:40.294 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:39 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:39.918+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:40.294 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:39 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:39.919+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:40.294 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:40 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:40.111+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:40.294 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:40 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:40.111+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:40.586 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:40 smithi169 podman[69980]: 2024-04-25 00:43:40.376734923 +0000 UTC m=+5.879423016 container cleanup e5491819dbec2c8423a3fdfcfc62435e7e0bced2bb1891203d0786a04a39128a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T00:43:40.586 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:40 smithi169 bash[69980]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5 2024-04-25T00:43:40.714 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:40 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:40.418+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:40.714 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:40 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:40.418+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:40.714 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:40 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:40.418+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:40.889 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:40 smithi169 podman[69997]: 2024-04-25 00:43:40.586817465 +0000 UTC m=+0.589085075 container remove e5491819dbec2c8423a3fdfcfc62435e7e0bced2bb1891203d0786a04a39128a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T00:43:41.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:40 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:40.713+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:41.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:40 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:40.713+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:41.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:41.046+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:41.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:41.046+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:41.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:41.046+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:41.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:40.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:41.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:40.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:41.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:40 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:40.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:41.318 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:40 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:40.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:41.318 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:40 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:40.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:41.318 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:41 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:41.120+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:41.318 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:41 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:41.120+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:41.318 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:41 smithi169 podman[70081]: 2024-04-25 00:43:40.903286666 +0000 UTC m=+0.018850432 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:41.318 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:41 smithi169 podman[70081]: 2024-04-25 00:43:41.02121832 +0000 UTC m=+0.136782111 container create 1e31c7c6196e6d5f8374d0bbbfc8d9ad8ce064751466ab8be7e618a1f786e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T00:43:41.319 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:41 smithi169 podman[70081]: 2024-04-25 00:43:41.318131144 +0000 UTC m=+0.433694904 container init 1e31c7c6196e6d5f8374d0bbbfc8d9ad8ce064751466ab8be7e618a1f786e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:43:41.612 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:41 smithi169 podman[70081]: 2024-04-25 00:43:41.321202942 +0000 UTC m=+0.436766692 container start 1e31c7c6196e6d5f8374d0bbbfc8d9ad8ce064751466ab8be7e618a1f786e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:43:41.612 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:41 smithi169 podman[70081]: 2024-04-25 00:43:41.418054655 +0000 UTC m=+0.533618404 container attach 1e31c7c6196e6d5f8374d0bbbfc8d9ad8ce064751466ab8be7e618a1f786e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:41.874 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 00:43:41 smithi169 podman[70081]: 2024-04-25 00:43:41.612290747 +0000 UTC m=+0.727854497 container died 1e31c7c6196e6d5f8374d0bbbfc8d9ad8ce064751466ab8be7e618a1f786e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:43:41.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:41 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:41.703+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:41.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:41 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:41.703+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:41.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:41 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:41.462+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:41.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:41 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:41.462+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:41.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:41 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:41.462+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:42.139 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:41 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:41.872+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:42.139 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:41 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:41.872+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:42.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:42 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:42.076+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:42.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:42 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:42.076+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:42.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T00:43:42.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:42.080+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:42.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:42.080+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:42.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:42.080+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:42.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:41.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:42.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:41.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:42.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:41 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:41.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:42.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:42.303 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T00:43:42.303 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T00:43:42.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.6 2024-04-25T00:43:42.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 systemd[1]: Stopping Ceph osd.6 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:42.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:42.633+0000 7f0a3424a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:42.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:42 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:42.718+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:42.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:42 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:42.718+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:42.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:42 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:42.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:42.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:42 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:42.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:42.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:42 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:42.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:42.936 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:42.633+0000 7f0a3424a700 -1 osd.6 94 *** Got signal Terminated *** 2024-04-25T00:43:42.936 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:42.633+0000 7f0a3424a700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:43.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:43.109+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:43.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:43.109+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:43.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:43.109+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:43.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:42.935+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:43.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:42.935+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:43.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:42 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:42.935+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:43.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:43.104+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:43.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:43.104+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:43.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:42 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:42.899+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:43.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:42 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:42.899+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:43.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:43.731+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:43.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:43.731+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:43.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:43.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:43.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:43.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:43.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:43 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:43.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:44.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:44.074+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:44.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:44.074+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:44.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:44.074+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:44.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:43.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:44.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:43.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:44.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:43 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:43.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:44.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:43.937+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:44.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:43 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:43.937+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:44.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:44 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:44.119+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:44.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:44 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:44.119+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:44.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:44 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:44.119+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:44.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:44 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:44.758+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:44.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:44 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:44.758+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:44.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:44 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:44.560+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:44.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:44 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:44.560+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:44.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:44 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:44.560+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:45.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:45.079+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:45.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:45.079+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:45.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:45.079+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:45.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:44.965+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:45.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:44.965+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:45.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:44 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:44.965+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:45.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:45.136+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:45.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:45.136+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:45.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:45.136+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:45.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:44 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:44.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:45.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:44 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:44.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:45.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:44 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:44.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:45.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:45 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:45.732+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:45.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:45 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:45.732+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:45.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:45 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:45.605+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:45.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:45 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:45.605+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:45.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:45 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:45.605+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:45.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:45 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:45.605+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:46.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:45.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:46.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:45.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:46.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:45.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:46.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:46.122+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:46.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:46.122+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:46.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:46.122+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:46.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:46.122+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:20.738788+0000 front 2024-04-25T00:43:20.738888+0000 (oldest deadline 2024-04-25T00:43:45.438740+0000) 2024-04-25T00:43:46.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:46 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:46.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:46.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:46 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:46.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:46.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:46 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:46.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:46.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:45.946+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:46.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:45.946+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:46.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:45 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:45.946+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:46.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:46 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:46.714+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:46.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:46 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:46.714+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:46.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:46 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:46.564+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:46.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:46 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:46.564+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:46.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:46 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:46.564+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:46.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:46 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:46.564+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:47.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:46.914+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:47.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:46.914+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:47.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:46 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:46.914+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:47.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:47.162+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:58.233851+0000 front 2024-04-25T00:42:58.233915+0000 (oldest deadline 2024-04-25T00:43:24.134065+0000) 2024-04-25T00:43:47.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:47.162+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:05.235535+0000 front 2024-04-25T00:43:05.235499+0000 (oldest deadline 2024-04-25T00:43:30.435735+0000) 2024-04-25T00:43:47.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:47.162+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:13.837684+0000 front 2024-04-25T00:43:13.837668+0000 (oldest deadline 2024-04-25T00:43:39.137394+0000) 2024-04-25T00:43:47.273 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6[54923]: 2024-04-25T00:43:47.162+0000 7f0a3083e700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:20.738788+0000 front 2024-04-25T00:43:20.738888+0000 (oldest deadline 2024-04-25T00:43:45.438740+0000) 2024-04-25T00:43:47.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:47 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:47.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:47.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:47 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:47.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:47.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:47 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:47.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:47.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:46 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:46.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:47.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:46 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:46.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:47.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:46 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:46.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:47.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:47 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:47.714+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:47.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:47 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:47.714+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:47.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:47 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:47.545+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:47.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:47 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:47.545+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:47.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:47 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:47.545+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:47.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:47 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:47.545+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:48.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:47 smithi031 podman[86579]: 2024-04-25 00:43:47.969446702 +0000 UTC m=+5.456099112 container died dd48adbfc11f6301fb2fbb814d052ab9aa791ee1f3f135be338eacebc71a09fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:43:48.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:47.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:48.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:47.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:48.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:47.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:48.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:47 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:47.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:48.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:48 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:48.091+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:48.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:48 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:48.091+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:48.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:48 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:48.091+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:48.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:47 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:47.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:48.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:47 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:47.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:48.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:47 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:47.923+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:48.585 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:48 smithi031 podman[86579]: 2024-04-25 00:43:48.384256989 +0000 UTC m=+5.870909412 container cleanup dd48adbfc11f6301fb2fbb814d052ab9aa791ee1f3f135be338eacebc71a09fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:43:48.585 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:48 smithi031 bash[86579]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:48.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:48.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:48.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:48.516+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:48.516+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:48.516+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:48.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:48.516+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:48.924 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:48 smithi031 podman[86596]: 2024-04-25 00:43:48.585650999 +0000 UTC m=+0.610047056 container remove dd48adbfc11f6301fb2fbb814d052ab9aa791ee1f3f135be338eacebc71a09fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-25T00:43:49.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:49 smithi031 podman[86687]: 2024-04-25 00:43:48.917753576 +0000 UTC m=+0.018484722 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:49.272 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:49 smithi031 podman[86687]: 2024-04-25 00:43:49.036573377 +0000 UTC m=+0.137304523 container create 34072ce3669e496341575b831f5f308f984e4d032bf4dd93b29b47f8974f6b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T00:43:49.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:48 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:48.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:49.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:48 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:48.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:49.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:48 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:48.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:49.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:48 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:48.924+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:49.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:49.060+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:49.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:49.060+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:49.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:49.060+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:49.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:48 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:48.948+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:49.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:48 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:48.948+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:49.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:48 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:48.948+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:49.577 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:49 smithi031 podman[86687]: 2024-04-25 00:43:49.273226811 +0000 UTC m=+0.373957956 container init 34072ce3669e496341575b831f5f308f984e4d032bf4dd93b29b47f8974f6b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T00:43:49.577 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:49 smithi031 podman[86687]: 2024-04-25 00:43:49.276746026 +0000 UTC m=+0.377477176 container start 34072ce3669e496341575b831f5f308f984e4d032bf4dd93b29b47f8974f6b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:43:49.578 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:49 smithi031 podman[86687]: 2024-04-25 00:43:49.414354082 +0000 UTC m=+0.515085242 container attach 34072ce3669e496341575b831f5f308f984e4d032bf4dd93b29b47f8974f6b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T00:43:49.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:49.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:49.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:49.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:49.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:49.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:49.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:49.505+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:49.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:49.505+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:49.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:49.505+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:49.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:49 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:49.505+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:49.909 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 00:43:49 smithi031 podman[86687]: 2024-04-25 00:43:49.578053436 +0000 UTC m=+0.678784597 container died 34072ce3669e496341575b831f5f308f984e4d032bf4dd93b29b47f8974f6b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:43:50.193 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:49 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:49.930+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:50.193 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:49 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:49.930+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:50.193 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:49 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:49.930+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:50.193 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:49 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:49.930+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:50.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T00:43:50.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:50.259 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T00:43:50.259 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T00:43:50.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.7 2024-04-25T00:43:50.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:50 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:50.069+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:50.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:50 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:50.069+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:50.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:50 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:50.069+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:50.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:49.998+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:50.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:49.998+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:50.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:49 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:49.998+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:50.548 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 systemd[1]: Stopping Ceph osd.7 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:50.547+0000 7fe81dcbe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:50.547+0000 7fe81dcbe700 -1 osd.7 94 *** Got signal Terminated *** 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:50.547+0000 7fe81dcbe700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:50.716+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:50.716+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:50.716+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:50.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:50.550+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:50.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:50.550+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:50.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:50.550+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:50.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:50 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:50.550+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:51.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:50.889+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:51.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:50.889+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:51.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:50.889+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:51.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:50 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:50.889+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:51.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:51 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:51.069+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:51.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:51 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:51.069+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:51.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:51 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:51.069+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:51.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:51 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:51.019+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:51.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:51 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:51.019+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:51.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:51 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:51.019+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:51.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:51.694+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:51.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:51.694+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:51.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:51.694+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:51.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:51.537+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:51.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:51.537+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:51.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:51.537+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:51.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:51 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:51.537+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:52.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:51 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:51.841+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:52.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:51 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:51.841+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:52.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:51 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:51.841+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:52.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:51 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:51.841+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:52.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:52 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:52.106+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:52.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:52 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:52.106+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:52.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:52 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:52.106+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:52.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:52 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:52.005+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:52.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:52 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:52.005+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:52.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:52 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:52.005+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:52.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:52.693+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:52.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:52.693+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:52.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:52.693+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:52.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:52.513+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:52.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:52.513+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:52.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:52.513+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:52.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:52.513+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:53.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:52 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:52.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:53.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:52 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:52.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:53.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:52 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:52.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:53.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:52 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:52.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:53.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:53 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:53.075+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:53.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:53 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:53.075+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:53.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:53 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:53.075+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:53.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:53 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:53.052+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:53.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:53 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:53.052+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:53.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:53 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:53.052+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:53.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:53.692+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:53.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:53.692+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:53.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:53.692+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:53.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:53.491+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:53.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:53.491+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:53.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:53.491+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:53.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:53 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:53.491+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:54.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:53 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:53.846+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:54.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:53 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:53.846+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:54.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:53 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:53.846+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:54.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:53 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:53.846+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:54.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:54.092+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:54.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:54.092+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:54.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:54.092+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:54.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:54.048+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:54.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:54.048+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:54.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:54.048+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:54.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:54 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:54.048+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:43:54.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:54.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:04.138642+0000 front 2024-04-25T00:43:04.138491+0000 (oldest deadline 2024-04-25T00:43:30.038529+0000) 2024-04-25T00:43:54.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:54.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:14.739657+0000 front 2024-04-25T00:43:14.739720+0000 (oldest deadline 2024-04-25T00:43:38.239512+0000) 2024-04-25T00:43:54.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:54.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.241623+0000 front 2024-04-25T00:43:22.241496+0000 (oldest deadline 2024-04-25T00:43:48.141408+0000) 2024-04-25T00:43:54.895 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7[50765]: 2024-04-25T00:43:54.681+0000 7fe8162aa700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:28.141804+0000 front 2024-04-25T00:43:28.141834+0000 (oldest deadline 2024-04-25T00:43:54.042028+0000) 2024-04-25T00:43:54.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:54.506+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:54.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:54.506+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:54.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:54.506+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:54.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:54.506+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:54.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:54 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:54.506+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:43:55.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:54.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:55.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:54.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:55.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:54.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:55.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:54 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:54.872+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:55.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:55.082+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:55.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:55.082+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:55.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:55.082+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:55.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:55.043+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:55.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:55.043+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:55.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:55.043+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:55.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:55 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:55.043+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:43:55.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:55 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:55.520+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:55.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:55 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:55.520+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:55.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:55 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:55.520+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:55.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:55 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:55.520+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:55.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:55 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:55.520+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:43:56.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:55 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:55.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:56.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:55 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:55.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:56.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:55 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:55.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:56.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:55 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:55.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:56.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:56.090+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:56.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:56.090+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:56.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:56.090+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:56.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:56.090+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:43:56.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:56.058+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:56.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:56.058+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:56.391 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:56.058+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:56.391 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:56 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:56.058+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:43:56.395 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:56 smithi040 podman[68365]: 2024-04-25 00:43:56.087072543 +0000 UTC m=+5.633975944 container died 3870ed79d0e1dcc1693bdf108be7947f3cebbdc9570351d5471bd7fe89f5429a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:43:56.857 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:56 smithi040 podman[68365]: 2024-04-25 00:43:56.576022447 +0000 UTC m=+6.122925845 container cleanup 3870ed79d0e1dcc1693bdf108be7947f3cebbdc9570351d5471bd7fe89f5429a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:43:56.857 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:56 smithi040 bash[68365]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7 2024-04-25T00:43:56.858 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:56.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:56.858 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:56.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:56.858 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:56.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:56.858 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:56.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:56.858 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:56.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:43:57.145 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:56 smithi040 podman[68382]: 2024-04-25 00:43:56.857604186 +0000 UTC m=+0.760959237 container remove 3870ed79d0e1dcc1693bdf108be7947f3cebbdc9570351d5471bd7fe89f5429a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T00:43:57.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:56 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:56.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:57.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:56 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:56.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:57.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:56 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:56.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:57.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:56 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:56.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:57.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:57.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:57.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:57.127+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:57.046+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:57.046+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:57.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:57.046+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:57.391 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:57 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:57.046+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:43:57.456 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:57 smithi040 podman[68471]: 2024-04-25 00:43:57.109101122 +0000 UTC m=+0.018734252 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:43:57.456 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:57 smithi040 podman[68471]: 2024-04-25 00:43:57.21695081 +0000 UTC m=+0.126583985 container create 6cec1f8c6c435e31cbcf3733ac332892eac721069ecaee7d7b00be03955f114b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:43:57.456 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:57 smithi040 podman[68471]: 2024-04-25 00:43:57.44572069 +0000 UTC m=+0.355353819 container init 6cec1f8c6c435e31cbcf3733ac332892eac721069ecaee7d7b00be03955f114b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T00:43:57.457 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:57 smithi040 podman[68471]: 2024-04-25 00:43:57.448849115 +0000 UTC m=+0.358482245 container start 6cec1f8c6c435e31cbcf3733ac332892eac721069ecaee7d7b00be03955f114b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T00:43:57.748 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:57 smithi040 podman[68471]: 2024-04-25 00:43:57.546124021 +0000 UTC m=+0.455757165 container attach 6cec1f8c6c435e31cbcf3733ac332892eac721069ecaee7d7b00be03955f114b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:43:57.748 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:57 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:57.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:57.748 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:57 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:57.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:57.748 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:57 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:57.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:57.748 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:57 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:57.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:57.749 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:57 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:57.455+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:43:58.033 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 00:43:57 smithi040 podman[68471]: 2024-04-25 00:43:57.748368649 +0000 UTC m=+0.658001788 container died 6cec1f8c6c435e31cbcf3733ac332892eac721069ecaee7d7b00be03955f114b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:43:58.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:57 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:57.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:58.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:57 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:57.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:58.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:57 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:57.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:58.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:57 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:57.897+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.093+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:58.054+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:58.054+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:58.054+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:58.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:58.054+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:43:58.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T00:43:58.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:43:58.532 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T00:43:58.532 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T00:43:58.532 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.8 2024-04-25T00:43:58.759 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:58 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:58.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:58.759 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:58 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:58.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:58.759 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:58 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:58.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:58.759 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:58 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:58.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:58.759 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:58 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:58.502+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:43:58.865 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 systemd[1]: Stopping Ceph osd.8 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:43:59.139 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.007+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.007+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.007+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.007+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.865+0000 7f0245f35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.865+0000 7f0245f35700 -1 osd.8 94 *** Got signal Terminated *** 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:58 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:58.865+0000 7f0245f35700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:59.089+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:59.089+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:59.089+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:43:59.140 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:43:59.089+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:43:59.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:58 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:58.849+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:43:59.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:58 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:58.849+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:43:59.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:58 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:58.849+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:43:59.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:58 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:58.849+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:43:59.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:59 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:59.510+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:43:59.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:59 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:59.510+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:43:59.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:59 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:59.510+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:43:59.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:59 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:59.510+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:43:59.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:43:59 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:43:59.510+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:00.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:59 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:59.845+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:00.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:59 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:59.845+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:00.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:59 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:59.845+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:00.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:43:59 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:43:59.845+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:00.066+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:00.066+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:00.066+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:00.066+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:00.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:43:59 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:43:59.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:00.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:00.504+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:00.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:00.504+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:00.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:00.504+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:00.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:00.504+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:00.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:00.504+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:01.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:00 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:00.895+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:01.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:00 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:00.895+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:01.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:00 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:00.895+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:01.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:00 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:00.895+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:01.291 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:01.037+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:44:01.291 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:01.037+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:01.037+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:01.037+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:00.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:00.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:00.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:00.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:01.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:00 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:00.995+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:01.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:01 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:01.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:01.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:01 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:01.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:01.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:01 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:01.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:01.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:01 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:01.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:01.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:01 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:01.529+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:02.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:01 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:01.896+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:02.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:01 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:01.896+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:02.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:01 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:01.896+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:02.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:01 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:01.896+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:02.057+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:02.057+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:02.057+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:02.057+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:01.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:01.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:02.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:01.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:02.312 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:01.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:02.312 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:01 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:01.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:02.797 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:02.542+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:02.797 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:02.542+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:02.797 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:02.542+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:02.797 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:02.542+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:02.797 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:02.542+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:02.797 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:02 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:02.542+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:03.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:02 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:02.927+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:03.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:02 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:02.927+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:03.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:02 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:02.927+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:03.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:02 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:02.927+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:03.389 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:03.035+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:07.222764+0000 front 2024-04-25T00:43:07.222857+0000 (oldest deadline 2024-04-25T00:43:31.922767+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:03.035+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.923270+0000 front 2024-04-25T00:43:11.923248+0000 (oldest deadline 2024-04-25T00:43:37.823333+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:03.035+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.925321+0000 front 2024-04-25T00:43:22.925288+0000 (oldest deadline 2024-04-25T00:43:44.025303+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:03.035+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.426752+0000 front 2024-04-25T00:43:30.426862+0000 (oldest deadline 2024-04-25T00:43:55.726692+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8[52047]: 2024-04-25T00:44:03.035+0000 7f0242529700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.827793+0000 front 2024-04-25T00:43:36.827836+0000 (oldest deadline 2024-04-25T00:44:02.127880+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:02.938+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:02.938+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:02.938+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:02.938+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:03.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:02 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:02.938+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:03.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:03 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:03.546+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:03.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:03 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:03.546+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:03.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:03 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:03.546+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:03.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:03 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:03.546+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:03.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:03 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:03.546+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:03.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:03 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:03.546+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:04.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:03 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:03.967+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:04.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:03 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:03.967+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:04.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:03 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:03.967+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:04.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:03 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:03.967+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:04.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:03 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:03.967+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:04.383 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:03.912+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:04.384 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:03.912+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:04.384 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:03.912+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:04.384 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:03.912+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:04.384 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:03 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:03.912+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:04.639 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:04 smithi169 podman[70187]: 2024-04-25 00:44:04.383777291 +0000 UTC m=+5.662141869 container died 58cc51c66e505f4788b7ecedcc346bdb1f157f9cf7b14344270b3fd7707cb5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T00:44:04.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:04 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:04.593+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:04.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:04 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:04.593+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:04.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:04 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:04.593+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:04.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:04 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:04.593+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:04.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:04 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:04.593+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:04.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:04 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:04.593+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:05.016 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:04 smithi169 podman[70187]: 2024-04-25 00:44:04.740269585 +0000 UTC m=+6.018634159 container cleanup 58cc51c66e505f4788b7ecedcc346bdb1f157f9cf7b14344270b3fd7707cb5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-25T00:44:05.016 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:04 smithi169 bash[70187]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8 2024-04-25T00:44:05.017 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:04.866+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:05.017 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:04.866+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:05.017 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:04.866+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:05.017 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:04.866+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:05.017 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:04.866+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:05.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:04 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:04.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:05.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:04 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:04.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:05.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:04 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:04.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:05.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:04 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:04.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:05.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:04 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:04.931+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:05.368 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70216]: 2024-04-25 00:44:05.016611801 +0000 UTC m=+0.631414324 container remove 58cc51c66e505f4788b7ecedcc346bdb1f157f9cf7b14344270b3fd7707cb5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:44:05.636 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70306]: 2024-04-25 00:44:05.271455128 +0000 UTC m=+0.018134421 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:44:05.636 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70306]: 2024-04-25 00:44:05.392574259 +0000 UTC m=+0.139253544 container create 75e901bac8303211f8e8a53a1ed5001e8ca16170f6f4b4039578a8d119015ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70306]: 2024-04-25 00:44:05.636492782 +0000 UTC m=+0.383172065 container init 75e901bac8303211f8e8a53a1ed5001e8ca16170f6f4b4039578a8d119015ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70306]: 2024-04-25 00:44:05.639780553 +0000 UTC m=+0.386459838 container start 75e901bac8303211f8e8a53a1ed5001e8ca16170f6f4b4039578a8d119015ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70306]: 2024-04-25 00:44:05.728290234 +0000 UTC m=+0.474969533 container attach 75e901bac8303211f8e8a53a1ed5001e8ca16170f6f4b4039578a8d119015ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:05 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:05.874+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:05 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:05.875+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:05 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:05.875+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:05 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:05.875+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:05.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:05 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:05.875+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:05.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:05.635+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:05.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:05.635+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:05.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:05.635+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:05.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:05.635+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:05.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:05.635+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:05.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:05 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:05.635+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:06.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:05.925+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:06.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:05.925+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:06.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:05.925+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:06.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:05.925+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:06.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:05 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:05.925+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:06.297 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 00:44:05 smithi169 podman[70306]: 2024-04-25 00:44:05.940188973 +0000 UTC m=+0.686868250 container died 75e901bac8303211f8e8a53a1ed5001e8ca16170f6f4b4039578a8d119015ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T00:44:06.591 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T00:44:06.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:06.633 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T00:44:06.633 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T00:44:06.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.9 2024-04-25T00:44:06.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:06.631+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:06.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:06.631+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:06.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:06.631+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:06.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:06.631+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:06.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:06.631+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:06.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:06 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:06.631+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:06.960 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 systemd[1]: Stopping Ceph osd.9 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:07.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:07.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:07.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:07.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:07.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.959+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:07.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.961+0000 7efd5101b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:07.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.961+0000 7efd5101b700 -1 osd.9 94 *** Got signal Terminated *** 2024-04-25T00:44:07.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:06 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:06.961+0000 7efd5101b700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:44:07.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:06 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:06.891+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:07.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:06 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:06.891+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:07.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:06 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:06.891+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:07.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:06 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:06.891+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:07.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:06 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:06.891+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:07.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:07 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:07.632+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:07.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:07 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:07.632+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:07.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:07 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:07.632+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:07.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:07 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:07.632+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:07.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:07 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:07.632+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:07.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:07 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:07.632+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:08.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:07 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:07.869+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:08.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:07 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:07.869+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:08.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:07 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:07.869+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:08.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:07 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:07.869+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:08.141 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:07 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:07.869+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:08.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:07 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:07.962+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:08.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:07 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:07.962+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:08.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:07 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:07.962+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:08.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:07 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:07.962+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:08.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:07 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:07.962+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:08.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:08 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:08.584+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:08.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:08 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:08.584+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:08.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:08 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:08.584+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:08.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:08 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:08.584+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:08.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:08 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:08.584+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:08.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:08 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:08.584+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:09.139 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:08.870+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:09.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:08.870+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:09.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:08.870+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:09.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:08.870+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:09.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:08.870+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:09.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:08.974+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:09.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:08.974+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:09.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:08.974+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:09.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:08.974+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:09.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:08 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:08.974+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:09.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:09 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:09.562+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:09.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:09 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:09.562+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:09.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:09 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:09.562+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:09.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:09 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:09.562+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:09.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:09 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:09.562+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:09.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:09 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:09.562+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:10.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:09.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:10.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:09.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:10.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:09.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:10.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:09.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:10.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:09.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:10.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:09 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:09.961+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:44.430221+0000 front 2024-04-25T00:43:44.430175+0000 (oldest deadline 2024-04-25T00:44:09.130270+0000) 2024-04-25T00:44:10.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:09 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:09.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:10.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:09 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:09.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:10.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:09 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:09.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:10.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:09 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:09.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:10.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:09 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:09.909+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:10.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:10 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:10.594+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:10.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:10 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:10.594+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:10.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:10 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:10.595+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:10.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:10 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:10.595+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:10.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:10 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:10.595+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:10.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:10 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:10.595+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:11.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:11.010+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:56.722436+0000 front 2024-04-25T00:42:56.722497+0000 (oldest deadline 2024-04-25T00:43:20.822554+0000) 2024-04-25T00:44:11.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:11.010+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.524587+0000 front 2024-04-25T00:43:06.524493+0000 (oldest deadline 2024-04-25T00:43:32.424467+0000) 2024-04-25T00:44:11.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:11.010+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.226272+0000 front 2024-04-25T00:43:15.226302+0000 (oldest deadline 2024-04-25T00:43:39.326368+0000) 2024-04-25T00:44:11.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:11.010+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.227178+0000 front 2024-04-25T00:43:22.227216+0000 (oldest deadline 2024-04-25T00:43:47.527408+0000) 2024-04-25T00:44:11.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:11.010+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:38.629672+0000 front 2024-04-25T00:43:38.629468+0000 (oldest deadline 2024-04-25T00:44:03.329353+0000) 2024-04-25T00:44:11.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:11 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9[61830]: 2024-04-25T00:44:11.010+0000 7efd49607700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:44.430221+0000 front 2024-04-25T00:43:44.430175+0000 (oldest deadline 2024-04-25T00:44:09.130270+0000) 2024-04-25T00:44:11.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:10.928+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:11.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:10.928+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:11.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:10.928+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:11.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:10.928+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:11.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:10 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:10.928+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:11.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:11 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:11.587+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:12.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:11 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:11.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:12.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:11 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:11.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:12.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:11 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:11.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:12.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:11 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:11.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:12.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:11 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:11.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:12.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:11 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:11.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:12.578 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:12 smithi031 podman[86794]: 2024-04-25 00:44:12.315030991 +0000 UTC m=+5.477930929 container died 91a79568fae66be5d10ef6ac89397a49ce093a9cd69b89c52f9043a00efb0a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T00:44:12.853 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:12.854 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:12.854 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:12.854 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:12.854 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:12.854 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:12.854 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:12 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:12.600+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:12.970 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:12 smithi031 podman[86794]: 2024-04-25 00:44:12.669453828 +0000 UTC m=+5.832353765 container cleanup 91a79568fae66be5d10ef6ac89397a49ce093a9cd69b89c52f9043a00efb0a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T00:44:12.970 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:12 smithi031 bash[86794]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9 2024-04-25T00:44:13.272 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:12 smithi031 podman[86811]: 2024-04-25 00:44:12.970775932 +0000 UTC m=+0.657066185 container remove 91a79568fae66be5d10ef6ac89397a49ce093a9cd69b89c52f9043a00efb0a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T00:44:13.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:12.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:13.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:12.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:13.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:12.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:13.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:12.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:13.391 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:12.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:13.391 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:12.969+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:13.565 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:13 smithi031 podman[86906]: 2024-04-25 00:44:13.202312317 +0000 UTC m=+0.018695730 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:44:13.565 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:13 smithi031 podman[86906]: 2024-04-25 00:44:13.314749446 +0000 UTC m=+0.131132847 container create 084042d6f69558ddf8a18b810d495be8258c9ac6f8109b986eb42d24cd0a27e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T00:44:13.566 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:13 smithi031 podman[86906]: 2024-04-25 00:44:13.56564917 +0000 UTC m=+0.382032570 container init 084042d6f69558ddf8a18b810d495be8258c9ac6f8109b986eb42d24cd0a27e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:44:13.869 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:13 smithi031 podman[86906]: 2024-04-25 00:44:13.568927233 +0000 UTC m=+0.385310634 container start 084042d6f69558ddf8a18b810d495be8258c9ac6f8109b986eb42d24cd0a27e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T00:44:13.869 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:13 smithi031 podman[86906]: 2024-04-25 00:44:13.640479679 +0000 UTC m=+0.456863089 container attach 084042d6f69558ddf8a18b810d495be8258c9ac6f8109b986eb42d24cd0a27e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:13.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:13 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:13.556+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:14.159 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 00:44:13 smithi031 podman[86906]: 2024-04-25 00:44:13.869409897 +0000 UTC m=+0.685793315 container died 084042d6f69558ddf8a18b810d495be8258c9ac6f8109b986eb42d24cd0a27e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:44:14.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:13 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:13.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:14.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:13 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:13.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:14.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:13 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:13.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:14.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:13 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:13.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:14.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:13 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:13.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:14.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:13 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:13.930+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:14.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T00:44:14.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:14.511 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T00:44:14.511 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T00:44:14.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.10 2024-04-25T00:44:14.828 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.526+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 systemd[1]: Stopping Ceph osd.10 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.828+0000 7f8833688700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.828+0000 7f8833688700 -1 osd.10 94 *** Got signal Terminated *** 2024-04-25T00:44:14.829 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:14 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:14.828+0000 7f8833688700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:44:15.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:14 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:14.902+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:15.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:14 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:14.902+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:15.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:14 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:14.902+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:15.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:14 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:14.902+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:15.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:14 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:14.902+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:15.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:14 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:14.902+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:15.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:15 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:15.500+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:16.389 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:15 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:15.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:16.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:15 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:15.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:16.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:15 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:15.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:16.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:15 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:15.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:16.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:15 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:15.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:16.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:15 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:15.920+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:16.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:16 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:16.501+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:17.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:16.907+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:17.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:16.907+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:17.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:16.907+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:17.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:16.907+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:17.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:16.907+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:17.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:16.907+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:17.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:17.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:17.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:17.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:17.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:17.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:17.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:17 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:17.523+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:18.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:17 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:17.858+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:18.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:17 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:17.858+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:18.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:17 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:17.858+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:18.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:17 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:17.858+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:18.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:17 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:17.858+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:18.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:17 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:17.858+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:18.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:18.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:18.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:18.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:18.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:18.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:18.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:18 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:18.511+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:19.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:18 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:18.815+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:19.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:53.602104+0000 front 2024-04-25T00:42:53.602010+0000 (oldest deadline 2024-04-25T00:43:23.602546+0000) 2024-04-25T00:44:19.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T00:43:06.503505+0000 front 2024-04-25T00:43:06.503528+0000 (oldest deadline 2024-04-25T00:43:28.803719+0000) 2024-04-25T00:44:19.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:15.205390+0000 front 2024-04-25T00:43:15.205435+0000 (oldest deadline 2024-04-25T00:43:36.305367+0000) 2024-04-25T00:44:19.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:22.607572+0000 front 2024-04-25T00:43:22.607527+0000 (oldest deadline 2024-04-25T00:43:44.907447+0000) 2024-04-25T00:44:19.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:31.209347+0000 front 2024-04-25T00:43:31.209407+0000 (oldest deadline 2024-04-25T00:43:53.509439+0000) 2024-04-25T00:44:19.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:39.311015+0000 front 2024-04-25T00:43:39.310999+0000 (oldest deadline 2024-04-25T00:44:01.610853+0000) 2024-04-25T00:44:19.896 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:19 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10[57645]: 2024-04-25T00:44:19.465+0000 7f882fc7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:45.112052+0000 front 2024-04-25T00:43:45.111963+0000 (oldest deadline 2024-04-25T00:44:11.011781+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:20.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:19 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:19.789+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:20.587 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:20 smithi040 podman[68588]: 2024-04-25 00:44:20.325992267 +0000 UTC m=+5.606915834 container died 3a374bfc2905dbe492c6f5f396234884559612c0c0fcbe96d82ab8a6d0ec4533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T00:44:20.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:20 smithi040 podman[68588]: 2024-04-25 00:44:20.727844424 +0000 UTC m=+6.008767993 container cleanup 3a374bfc2905dbe492c6f5f396234884559612c0c0fcbe96d82ab8a6d0ec4533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:44:20.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:20 smithi040 bash[68588]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:21.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:20 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:20.833+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:21.279 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:20 smithi040 podman[68606]: 2024-04-25 00:44:20.962746621 +0000 UTC m=+0.628925032 container remove 3a374bfc2905dbe492c6f5f396234884559612c0c0fcbe96d82ab8a6d0ec4533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T00:44:21.591 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:21 smithi040 podman[68698]: 2024-04-25 00:44:21.280114942 +0000 UTC m=+0.112377468 container create e84e7595eba881a9d124a9750de2b65f6451b407b2fc8da8334c61b5d03a37cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:44:21.592 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:21 smithi040 podman[68698]: 2024-04-25 00:44:21.186694789 +0000 UTC m=+0.018957330 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:44:21.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:21 smithi040 podman[68698]: 2024-04-25 00:44:21.591456894 +0000 UTC m=+0.423719425 container init e84e7595eba881a9d124a9750de2b65f6451b407b2fc8da8334c61b5d03a37cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:44:21.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:21 smithi040 podman[68698]: 2024-04-25 00:44:21.595095736 +0000 UTC m=+0.427358271 container start e84e7595eba881a9d124a9750de2b65f6451b407b2fc8da8334c61b5d03a37cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T00:44:21.895 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:21 smithi040 podman[68698]: 2024-04-25 00:44:21.665896814 +0000 UTC m=+0.498159356 container attach e84e7595eba881a9d124a9750de2b65f6451b407b2fc8da8334c61b5d03a37cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:22.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:21 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:21.842+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:22.185 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 00:44:21 smithi040 podman[68698]: 2024-04-25 00:44:21.897432973 +0000 UTC m=+0.729695503 container died e84e7595eba881a9d124a9750de2b65f6451b407b2fc8da8334c61b5d03a37cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T00:44:22.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T00:44:22.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:22.496 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T00:44:22.496 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T00:44:22.496 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.11 2024-04-25T00:44:22.784 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 systemd[1]: Stopping Ceph osd.11 for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.783+0000 7f5d8b24a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.783+0000 7f5d8b24a700 -1 osd.11 94 *** Got signal Terminated *** 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.783+0000 7f5d8b24a700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:23.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:22 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:22.862+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:24.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:23 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:23.825+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:25.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:24 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:24.819+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:26.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:25 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:25.772+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T00:44:00.774887+0000 front 2024-04-25T00:44:00.774909+0000 (oldest deadline 2024-04-25T00:44:24.874832+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:27.140 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:26 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:26.757+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T00:44:00.774887+0000 front 2024-04-25T00:44:00.774909+0000 (oldest deadline 2024-04-25T00:44:24.874832+0000) 2024-04-25T00:44:28.096 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T00:42:57.665609+0000 front 2024-04-25T00:42:57.665689+0000 (oldest deadline 2024-04-25T00:43:21.165819+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T00:43:11.167663+0000 front 2024-04-25T00:43:11.167687+0000 (oldest deadline 2024-04-25T00:43:36.467779+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T00:43:19.968737+0000 front 2024-04-25T00:43:19.968771+0000 (oldest deadline 2024-04-25T00:43:44.068759+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T00:43:30.470517+0000 front 2024-04-25T00:43:30.470439+0000 (oldest deadline 2024-04-25T00:43:53.370351+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T00:43:36.271349+0000 front 2024-04-25T00:43:36.271321+0000 (oldest deadline 2024-04-25T00:44:00.371450+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T00:43:46.172850+0000 front 2024-04-25T00:43:46.172878+0000 (oldest deadline 2024-04-25T00:44:11.472845+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.40:6820 osd.7 since back 2024-04-25T00:43:52.573828+0000 front 2024-04-25T00:43:52.573845+0000 (oldest deadline 2024-04-25T00:44:18.473726+0000) 2024-04-25T00:44:28.097 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:27 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11[58932]: 2024-04-25T00:44:27.735+0000 7f5d8783e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T00:44:00.774887+0000 front 2024-04-25T00:44:00.774909+0000 (oldest deadline 2024-04-25T00:44:24.874832+0000) 2024-04-25T00:44:28.359 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:28 smithi169 podman[70411]: 2024-04-25 00:44:28.096648161 +0000 UTC m=+5.401963250 container died 1dcae73c7574f8c3ab9f4a7c7a93136865cc15bfc6ad9fe71e95df483b224c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:44:28.639 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:28 smithi169 podman[70411]: 2024-04-25 00:44:28.45822067 +0000 UTC m=+5.763535758 container cleanup 1dcae73c7574f8c3ab9f4a7c7a93136865cc15bfc6ad9fe71e95df483b224c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:44:28.640 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:28 smithi169 bash[70411]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11 2024-04-25T00:44:29.087 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:28 smithi169 podman[70429]: 2024-04-25 00:44:28.734871301 +0000 UTC m=+0.633312091 container remove 1dcae73c7574f8c3ab9f4a7c7a93136865cc15bfc6ad9fe71e95df483b224c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T00:44:29.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:29 smithi169 podman[70521]: 2024-04-25 00:44:28.990693899 +0000 UTC m=+0.017984595 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:44:29.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:29 smithi169 podman[70521]: 2024-04-25 00:44:29.093913346 +0000 UTC m=+0.121204033 container create 8ab1c8d098998e350ae80c65165c41393815bcfa4ca9a80071b316ff00d4d00d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:44:29.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:29 smithi169 podman[70521]: 2024-04-25 00:44:29.32133651 +0000 UTC m=+0.348627208 container init 8ab1c8d098998e350ae80c65165c41393815bcfa4ca9a80071b316ff00d4d00d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T00:44:29.390 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:29 smithi169 podman[70521]: 2024-04-25 00:44:29.324356711 +0000 UTC m=+0.351647406 container start 8ab1c8d098998e350ae80c65165c41393815bcfa4ca9a80071b316ff00d4d00d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:44:29.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:29 smithi169 podman[70521]: 2024-04-25 00:44:29.433360393 +0000 UTC m=+0.460651085 container attach 8ab1c8d098998e350ae80c65165c41393815bcfa4ca9a80071b316ff00d4d00d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T00:44:29.890 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 00:44:29 smithi169 podman[70521]: 2024-04-25 00:44:29.616174333 +0000 UTC m=+0.643465013 container died 8ab1c8d098998e350ae80c65165c41393815bcfa4ca9a80071b316ff00d4d00d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-25T00:44:30.185 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T00:44:30.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:30.223 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T00:44:30.223 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T00:44:30.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.a 2024-04-25T00:44:30.522 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:44:30 smithi031 systemd[1]: Stopping Ceph mds.a for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:31.022 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:44:30 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a[68213]: 2024-04-25T00:44:30.571+0000 7fa38724d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:31.022 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:44:30 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a[68213]: 2024-04-25T00:44:30.571+0000 7fa38724d700 -1 mds.a *** got signal Terminated *** 2024-04-25T00:44:34.846 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:44:34 smithi031 podman[87026]: 2024-04-25 00:44:34.582721342 +0000 UTC m=+4.161609667 container died a44ed774e70d7a2280a3c8e3b83d823878e64ef126abe289eddc02bc8fffe350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T00:44:35.231 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:44:34 smithi031 podman[87026]: 2024-04-25 00:44:34.937051371 +0000 UTC m=+4.515939696 container cleanup a44ed774e70d7a2280a3c8e3b83d823878e64ef126abe289eddc02bc8fffe350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:44:35.231 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 00:44:34 smithi031 bash[87026]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-a 2024-04-25T00:44:35.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T00:44:35.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:35.401 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T00:44:35.401 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T00:44:35.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.d 2024-04-25T00:44:35.733 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:35 smithi031 systemd[1]: Stopping Ceph mds.d for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:36.022 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:35 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d[67262]: 2024-04-25T00:44:35.732+0000 7fd894521700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:36.022 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:35 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d[67262]: 2024-04-25T00:44:35.732+0000 7fd894521700 -1 mds.d *** got signal Terminated *** 2024-04-25T00:44:39.216 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:38 smithi031 podman[87148]: 2024-04-25 00:44:38.875738716 +0000 UTC m=+3.250383524 container died 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T00:44:39.216 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:39 smithi031 podman[87148]: 2024-04-25 00:44:39.216069535 +0000 UTC m=+3.590714332 container cleanup 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-25T00:44:39.522 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:39 smithi031 bash[87148]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d 2024-04-25T00:44:39.522 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 00:44:39 smithi031 podman[87163]: 2024-04-25 00:44:39.392278371 +0000 UTC m=+0.512304907 container remove 1499cb49b90e3c27ed5cf0132c6c0b79513bea81f67b162ab3c8df9dd2ba2587 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:44:39.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T00:44:39.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:39.658 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T00:44:39.659 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T00:44:39.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.g 2024-04-25T00:44:39.978 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:44:39 smithi031 systemd[1]: Stopping Ceph mds.g for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:40.272 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:44:39 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g[67873]: 2024-04-25T00:44:39.977+0000 7f5bfcaa2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:40.272 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:44:39 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g[67873]: 2024-04-25T00:44:39.977+0000 7f5bfcaa2700 -1 mds.g *** got signal Terminated *** 2024-04-25T00:44:44.246 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:44:43 smithi031 podman[87270]: 2024-04-25 00:44:43.989789957 +0000 UTC m=+4.111155079 container died d24fd09ac27caefedb4bc5d74ce308b260cd1db76cde9a4dc3926090a9b46a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:44:44.522 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:44:44 smithi031 podman[87270]: 2024-04-25 00:44:44.344417931 +0000 UTC m=+4.465783052 container cleanup d24fd09ac27caefedb4bc5d74ce308b260cd1db76cde9a4dc3926090a9b46a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:44:44.522 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 00:44:44 smithi031 bash[87270]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-g 2024-04-25T00:44:44.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T00:44:44.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:44.773 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T00:44:44.774 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T00:44:44.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.j 2024-04-25T00:44:45.022 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:44 smithi031 systemd[1]: Stopping Ceph mds.j for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:45.522 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j[66914]: 2024-04-25T00:44:45.129+0000 7f2b71392700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:45.522 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:45 smithi031 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j[66914]: 2024-04-25T00:44:45.129+0000 7f2b71392700 -1 mds.j *** got signal Terminated *** 2024-04-25T00:44:47.391 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 podman[87391]: 2024-04-25 00:44:47.134201677 +0000 UTC m=+2.136252464 container died 7ca3afb6f2a61b5dfebba4ff0efc8fc7d5dae50783d065af5b26097e7f7991da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-25T00:44:47.657 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 podman[87391]: 2024-04-25 00:44:47.464798212 +0000 UTC m=+2.466848998 container cleanup 7ca3afb6f2a61b5dfebba4ff0efc8fc7d5dae50783d065af5b26097e7f7991da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T00:44:47.658 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 bash[87391]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j 2024-04-25T00:44:47.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T00:44:47.914 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 podman[87405]: 2024-04-25 00:44:47.657359444 +0000 UTC m=+0.517119346 container remove 7ca3afb6f2a61b5dfebba4ff0efc8fc7d5dae50783d065af5b26097e7f7991da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:44:47.914 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 systemd[1]: ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-25T00:44:47.914 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 systemd[1]: Stopped Ceph mds.j for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:44:47.914 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 00:44:47 smithi031 systemd[1]: ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.j.service: Consumed 2.091s CPU time. 2024-04-25T00:44:47.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:47.925 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T00:44:47.925 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T00:44:47.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.b 2024-04-25T00:44:48.229 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:44:47 smithi040 systemd[1]: Stopping Ceph mds.b for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:48.568 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:44:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b[63138]: 2024-04-25T00:44:48.227+0000 7f231e0fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:48.568 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:44:48 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b[63138]: 2024-04-25T00:44:48.227+0000 7f231e0fd700 -1 mds.b *** got signal Terminated *** 2024-04-25T00:44:51.463 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:44:51 smithi040 podman[68803]: 2024-04-25 00:44:51.157816202 +0000 UTC m=+3.038548813 container died f93e9953f001bb80ad27bfdaf443d8b68abda89bd1598f4587e4fbf52049c67b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:44:51.772 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:44:51 smithi040 podman[68803]: 2024-04-25 00:44:51.537486655 +0000 UTC m=+3.418219248 container cleanup f93e9953f001bb80ad27bfdaf443d8b68abda89bd1598f4587e4fbf52049c67b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:44:51.772 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 00:44:51 smithi040 bash[68803]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-b 2024-04-25T00:44:51.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T00:44:51.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:51.931 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T00:44:51.931 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T00:44:51.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.e 2024-04-25T00:44:52.145 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:52 smithi040 systemd[1]: Stopping Ceph mds.e for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:52.645 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e[62516]: 2024-04-25T00:44:52.257+0000 7fb3d0e5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:52.645 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:52 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e[62516]: 2024-04-25T00:44:52.257+0000 7fb3d0e5f700 -1 mds.e *** got signal Terminated *** 2024-04-25T00:44:55.408 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:55 smithi040 podman[68935]: 2024-04-25 00:44:55.153967476 +0000 UTC m=+3.002275737 container died d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T00:44:55.667 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:55 smithi040 podman[68935]: 2024-04-25 00:44:55.482927302 +0000 UTC m=+3.331235561 container cleanup d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:44:55.667 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:55 smithi040 bash[68935]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e 2024-04-25T00:44:55.921 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 00:44:55 smithi040 podman[68949]: 2024-04-25 00:44:55.667421583 +0000 UTC m=+0.510038180 container remove d18d84a04dc89480c0bc716f23fca8ef38d2391ed96e5a39ba5a562905aed9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T00:44:55.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T00:44:55.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:55.958 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T00:44:55.958 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T00:44:55.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.h 2024-04-25T00:44:56.271 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:44:56 smithi040 systemd[1]: Stopping Ceph mds.h for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:44:56.645 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:44:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h[62827]: 2024-04-25T00:44:56.269+0000 7ff0b0aeb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:44:56.645 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:44:56 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h[62827]: 2024-04-25T00:44:56.269+0000 7ff0b0aeb700 -1 mds.h *** got signal Terminated *** 2024-04-25T00:44:59.395 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:44:59 smithi040 podman[69056]: 2024-04-25 00:44:59.143310539 +0000 UTC m=+2.965750552 container died 1eaedcfead6aaf39d79b5f947eabdd94ce4489bc8a22a692dc006dd80ad7b243 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T00:44:59.771 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:44:59 smithi040 podman[69056]: 2024-04-25 00:44:59.503828696 +0000 UTC m=+3.326268705 container cleanup 1eaedcfead6aaf39d79b5f947eabdd94ce4489bc8a22a692dc006dd80ad7b243 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:44:59.771 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 00:44:59 smithi040 bash[69056]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-h 2024-04-25T00:44:59.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T00:44:59.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:44:59.962 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T00:44:59.962 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T00:44:59.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.k 2024-04-25T00:45:00.299 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:45:00 smithi040 systemd[1]: Stopping Ceph mds.k for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:45:00.645 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:45:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k[62196]: 2024-04-25T00:45:00.298+0000 7f0c61ac0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:00.645 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:45:00 smithi040 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k[62196]: 2024-04-25T00:45:00.298+0000 7f0c61ac0700 -1 mds.k *** got signal Terminated *** 2024-04-25T00:45:03.395 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:45:03 smithi040 podman[69179]: 2024-04-25 00:45:03.142760486 +0000 UTC m=+2.953091726 container died 28e2f54d7edda45090007d67cf880778731c1d4b2badfdbea90c85bf05208859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T00:45:03.766 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:45:03 smithi040 podman[69179]: 2024-04-25 00:45:03.498897002 +0000 UTC m=+3.309228241 container cleanup 28e2f54d7edda45090007d67cf880778731c1d4b2badfdbea90c85bf05208859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T00:45:03.767 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 00:45:03 smithi040 bash[69179]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-k 2024-04-25T00:45:03.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T00:45:03.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:03.961 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T00:45:03.961 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T00:45:03.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.c 2024-04-25T00:45:04.332 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:45:04 smithi169 systemd[1]: Stopping Ceph mds.c for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:45:04.639 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:45:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c[64064]: 2024-04-25T00:45:04.331+0000 7fbeb6808700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:04.640 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:45:04 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c[64064]: 2024-04-25T00:45:04.331+0000 7fbeb6808700 -1 mds.c *** got signal Terminated *** 2024-04-25T00:45:07.415 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:45:07 smithi169 podman[70626]: 2024-04-25 00:45:07.148064111 +0000 UTC m=+2.998682407 container died 0b0fc29d034e5e1aeeaed17b7dc9f120b1a210338672e0febd48bdc1fba19fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:45:07.715 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:45:07 smithi169 podman[70626]: 2024-04-25 00:45:07.489628876 +0000 UTC m=+3.340247164 container cleanup 0b0fc29d034e5e1aeeaed17b7dc9f120b1a210338672e0febd48bdc1fba19fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:45:07.716 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 00:45:07 smithi169 bash[70626]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-c 2024-04-25T00:45:07.921 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T00:45:07.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:07.955 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T00:45:07.955 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T00:45:07.955 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.f 2024-04-25T00:45:08.285 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:45:08 smithi169 systemd[1]: Stopping Ceph mds.f for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:45:08.639 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:45:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f[63748]: 2024-04-25T00:45:08.284+0000 7fb828e30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:08.640 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:45:08 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f[63748]: 2024-04-25T00:45:08.284+0000 7fb828e30700 -1 mds.f *** got signal Terminated *** 2024-04-25T00:45:11.639 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:45:11 smithi169 podman[70745]: 2024-04-25 00:45:11.155921673 +0000 UTC m=+2.980564788 container died ecbb7508dd8a3091f1fff239b7d30a5afee546723358fd8eefced9de2fbe7069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T00:45:12.086 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:45:11 smithi169 podman[70745]: 2024-04-25 00:45:11.8099918 +0000 UTC m=+3.634634918 container cleanup ecbb7508dd8a3091f1fff239b7d30a5afee546723358fd8eefced9de2fbe7069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:45:12.086 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 00:45:11 smithi169 bash[70745]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-f 2024-04-25T00:45:12.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T00:45:12.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:12.278 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T00:45:12.279 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T00:45:12.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.i 2024-04-25T00:45:12.597 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:12 smithi169 systemd[1]: Stopping Ceph mds.i for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:45:12.889 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: 2024-04-25T00:45:12.596+0000 7fd64e6a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:12.889 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:12 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i[64375]: 2024-04-25T00:45:12.596+0000 7fd64e6a6700 -1 mds.i *** got signal Terminated *** 2024-04-25T00:45:15.389 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 podman[70865]: 2024-04-25 00:45:15.136576958 +0000 UTC m=+2.638772912 container died 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T00:45:15.731 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 podman[70865]: 2024-04-25 00:45:15.555150051 +0000 UTC m=+3.057345993 container cleanup 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-25T00:45:15.731 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 bash[70865]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i 2024-04-25T00:45:15.980 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T00:45:16.001 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 podman[70879]: 2024-04-25 00:45:15.731257394 +0000 UTC m=+0.589181777 container remove 08f3ddf14c4ce7ac4591b607e3409aeeb55e2dff361bab6c559aaffc0602f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:45:16.001 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 systemd[1]: ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-25T00:45:16.001 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 systemd[1]: Stopped Ceph mds.i for 6c091b24-0299-11ef-bc93-c7b262605968. 2024-04-25T00:45:16.001 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 00:45:15 smithi169 systemd[1]: ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.i.service: Consumed 12.924s CPU time. 2024-04-25T00:45:16.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:16.011 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T00:45:16.012 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T00:45:16.012 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.l 2024-04-25T00:45:16.325 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:45:16 smithi169 systemd[1]: Stopping Ceph mds.l for 6c091b24-0299-11ef-bc93-c7b262605968... 2024-04-25T00:45:16.639 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:45:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l[63436]: 2024-04-25T00:45:16.324+0000 7f7a21c84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:16.640 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:45:16 smithi169 ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l[63436]: 2024-04-25T00:45:16.324+0000 7f7a21c84700 -1 mds.l *** got signal Terminated *** 2024-04-25T00:45:19.559 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:45:19 smithi169 podman[70986]: 2024-04-25 00:45:19.145252033 +0000 UTC m=+2.916540577 container died 2200eac577be8f9ab7c0bde2abb9fd559f78c5337d9666e5779e9be4fbb36690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:45:19.889 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:45:19 smithi169 podman[70986]: 2024-04-25 00:45:19.641808843 +0000 UTC m=+3.413097387 container cleanup 2200eac577be8f9ab7c0bde2abb9fd559f78c5337d9666e5779e9be4fbb36690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T00:45:19.890 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 00:45:19 smithi169 bash[70986]: ceph-6c091b24-0299-11ef-bc93-c7b262605968-mds-l 2024-04-25T00:45:20.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6c091b24-0299-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T00:45:20.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:20.101 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T00:45:20.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:45:20.301 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:45:22.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:45:23.062 INFO:teuthology.orchestra.run.smithi040.stdout:Deleting cluster with fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:45:25.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:45:25.810 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:45:28.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:45:28.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:45:28.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:45:28.276 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T00:45:28.278 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi031/crash 2024-04-25T00:45:28.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:45:28.317 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:45:28.317 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:45:28.319 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi040/crash 2024-04-25T00:45:28.320 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:45:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:tar: /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:45:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:45:28.357 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi169/crash 2024-04-25T00:45:28.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:45:28.391 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/6c091b24-0299-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:45:28.391 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:45:28.392 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T00:45:28.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T00:45:28.427 INFO:tasks.cephadm:Compressing logs... 2024-04-25T00:45:28.428 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:45:28.470 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:45:28.472 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:45:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:45:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:: No such file or directory 2024-04-25T00:45:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:45:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T00:45:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T00:45:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T00:45:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T00:45:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T00:45:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:45:28.507 INFO:teuthology.orchestra.run.smithi169.stderr: 93.0% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.1% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.7% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T00:45:28.508 INFO:teuthology.orchestra.run.smithi031.stderr: 82.8% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:45:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:45:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:: No such file or directory 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T00:45:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip 87.4% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr: 90.3% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.audit.log.gz/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.10.log: 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.h.log: 93.0% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr: 82.7%/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.e.log: -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:45:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T00:45:28.519 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T00:45:28.519 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T00:45:28.519 INFO:teuthology.orchestra.run.smithi169.stderr: 82.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:45:28.520 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:45:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T00:45:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-25T00:45:28.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T00:45:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:45:28.522 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:45:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:45:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:45:28.535 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T00:45:28.541 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:45:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:45:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:45:28.572 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:45:28.572 INFO:teuthology.orchestra.run.smithi169.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:45:28.572 INFO:teuthology.orchestra.run.smithi031.stderr: 94.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T00:45:28.580 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T00:45:28.580 INFO:teuthology.orchestra.run.smithi040.stderr: 93.3% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T00:45:28.583 INFO:teuthology.orchestra.run.smithi040.stderr: 94.5% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T00:45:28.601 INFO:teuthology.orchestra.run.smithi040.stderr: 94.7% 94.5% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T00:45:28.601 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T00:45:28.601 INFO:teuthology.orchestra.run.smithi031.stderr: 93.5% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T00:45:28.637 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T00:45:28.731 INFO:teuthology.orchestra.run.smithi169.stderr: 94.5% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T00:45:28.795 INFO:teuthology.orchestra.run.smithi031.stderr: 90.2% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T00:45:28.834 INFO:teuthology.orchestra.run.smithi040.stderr: 92.3% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T00:45:28.834 INFO:teuthology.orchestra.run.smithi031.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-25T00:45:28.871 INFO:teuthology.orchestra.run.smithi169.stderr: 91.9% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T00:45:29.215 INFO:teuthology.orchestra.run.smithi031.stderr: 92.5% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T00:45:29.238 INFO:teuthology.orchestra.run.smithi169.stderr: 92.8% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T00:45:29.287 INFO:teuthology.orchestra.run.smithi031.stderr: 92.1% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T00:45:29.372 INFO:teuthology.orchestra.run.smithi169.stderr: 92.2% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T00:45:29.589 INFO:teuthology.orchestra.run.smithi031.stderr: 90.4% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T00:45:29.697 INFO:teuthology.orchestra.run.smithi031.stderr: 93.0% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T00:45:29.892 INFO:teuthology.orchestra.run.smithi040.stderr: 92.1% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T00:45:29.901 INFO:teuthology.orchestra.run.smithi031.stderr: 92.8% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T00:45:29.903 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-04-25T00:45:29.903 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m1.420s 2024-04-25T00:45:29.903 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m5.785s 2024-04-25T00:45:29.903 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.183s 2024-04-25T00:45:29.906 INFO:teuthology.orchestra.run.smithi040.stderr: 92.7% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T00:45:30.043 INFO:teuthology.orchestra.run.smithi169.stderr: 91.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T00:45:30.253 INFO:teuthology.orchestra.run.smithi040.stderr: 91.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T00:45:30.258 INFO:teuthology.orchestra.run.smithi169.stderr: 92.9% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T00:45:30.341 INFO:teuthology.orchestra.run.smithi040.stderr: 92.2% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T00:45:30.343 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-25T00:45:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m1.856s 2024-04-25T00:45:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m6.784s 2024-04-25T00:45:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.231s 2024-04-25T00:45:30.873 INFO:teuthology.orchestra.run.smithi169.stderr: 91.6% -- replaced with /var/log/ceph/6c091b24-0299-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T00:45:30.875 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-25T00:45:30.875 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m2.389s 2024-04-25T00:45:30.875 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m7.746s 2024-04-25T00:45:30.875 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.280s 2024-04-25T00:45:30.875 INFO:tasks.cephadm:Archiving logs... 2024-04-25T00:45:30.876 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi031/log 2024-04-25T00:45:30.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:45:31.769 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi040/log 2024-04-25T00:45:31.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:45:32.859 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi169/log 2024-04-25T00:45:32.859 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:45:34.028 INFO:tasks.cephadm:Removing cluster... 2024-04-25T00:45:34.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --force 2024-04-25T00:45:34.228 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:45:34.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --force 2024-04-25T00:45:34.682 INFO:teuthology.orchestra.run.smithi040.stdout:Deleting cluster with fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:45:34.960 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6c091b24-0299-11ef-bc93-c7b262605968 --force 2024-04-25T00:45:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 6c091b24-0299-11ef-bc93-c7b262605968 2024-04-25T00:45:35.429 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T00:45:35.429 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:45:35.447 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:45:35.464 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:45:35.479 INFO:tasks.cephadm:Teardown complete 2024-04-25T00:45:35.479 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:45:35.492 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:45:35.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:45:35.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:45:35.506 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:45:35.619 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:45:35.629 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:45:35.629 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:45:35.632 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:45:35.634 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:45:35.645 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:45:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:45:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:45:35.650 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:45:35.650 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-25T00:45:35.650 INFO:teuthology.orchestra.run.smithi031.stdout:^- hv01.front.sepia.ceph.com 2 6 377 14 +1002us[+1004us] +/- 579ms 2024-04-25T00:45:35.651 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -74us[ -73us] +/- 60ms 2024-04-25T00:45:35.651 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +120us[ +122us] +/- 68ms 2024-04-25T00:45:35.651 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:45:35.652 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:45:35.652 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T00:45:35.652 INFO:teuthology.orchestra.run.smithi169.stdout:^- hv01.front.sepia.ceph.com 2 6 377 17 +993us[ +993us] +/- 579ms 2024-04-25T00:45:35.653 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -74us[ -67us] +/- 59ms 2024-04-25T00:45:35.653 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +81us[ +81us] +/- 68ms 2024-04-25T00:45:35.653 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:45:35.654 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:45:35.654 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-25T00:45:35.654 INFO:teuthology.orchestra.run.smithi040.stdout:^- hv01.front.sepia.ceph.com 2 6 377 15 +1015us[+1015us] +/- 579ms 2024-04-25T00:45:35.654 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -67us[ -64us] +/- 60ms 2024-04-25T00:45:35.654 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +96us[ +99us] +/- 68ms 2024-04-25T00:45:35.654 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:45:35.654 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:45:35.713 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:45:35.714 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:45:35.724 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:45:35.802 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:45:35.873 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:45:35.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:45:36.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:45:36.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:45:36.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:45:36.133 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:45:36.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:45:36.160 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:45:36.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:45:36.183 INFO:teuthology.task.internal:Duration was 1920.799261 seconds 2024-04-25T00:45:36.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:45:36.194 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:45:36.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:45:36.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:45:36.199 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:45:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:45:36.236 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:45:36.240 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:45:36.698 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:45:36.698 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:45:36.699 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:45:36.723 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T00:45:36.723 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:45:36.749 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:45:36.749 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:45:36.772 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:45:36.773 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:45:36.777 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:45:36.791 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:45:36.844 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:45:36.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:45:36.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:45:36.864 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:45:37.242 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:45:37.255 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:45:37.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:45:37.280 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:45:37.305 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:45:37.330 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:45:37.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:45:37.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:45:37.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:45:37.364 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-04-25T00:45:37.370 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-25T00:45:37.393 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T00:45:37.449 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:45:37.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:45:37.463 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:45:37.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:45:37.479 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:45:37.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:45:37.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:45:37.521 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:45:37.522 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi031 2024-04-25T00:45:37.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:45:37.564 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi040 2024-04-25T00:45:37.565 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:45:37.606 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672254/remote/smithi169 2024-04-25T00:45:37.606 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:45:37.643 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:45:37.644 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:45:37.646 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:45:37.649 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:45:37.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:45:37.706 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:45:37.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:45:37.718 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:45:37.718 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:45:37.721 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:45:37.723 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:45:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:45 /home/ubuntu/cephtest 2024-04-25T00:45:37.736 INFO:teuthology.orchestra.run.smithi040.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:45 /home/ubuntu/cephtest 2024-04-25T00:45:37.748 INFO:teuthology.orchestra.run.smithi169.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:45 /home/ubuntu/cephtest 2024-04-25T00:45:37.749 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:45:37.761 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:45:37.787 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 1920.7992606163025 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:45:37.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:45:37.867 INFO:teuthology.run:pass